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INTRODUCTION 


Research  laboratories  for  the  military  have  been  exploring  the  potential 
of  using  new  types  of  tests  in  military  selection  and  classification.  For  more 
than  the  last  five  years,  the  Navy  Personnel  Research  and  Development  Center 
(NPRDC)  has  focused  on  developing  new  computerized  ability  tests.  These  tests 
measure  abilities  that  are  not  measured  by  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB),  currently  used  for  selection  and  classification  of  military 
applicants.  NPRDC  has  explored  over  30  tests  and  developed  an  experimental 
battery.  They  have  also  included  some  of  these  tests  in  a  joint-service  battery. 
NPRDC  and  its  contractor  have  been  collecting  data  to  evaluate  the  predictive 
validity  of  this  joint-service  battery,  referred  to  as  Enhanced  Computer 

Administered  Tests  (ECAT). 

To  evaluate  the  validity  of  a  test,  the  contractor  has  collected  two  types 
of  data:  1)  test  scores  on  the  predictor  tests,  specifically,  the  new 

computerized  tests,  the  CAT-ASVAB  and  the  P&P-ASVAB,  and  2)  data  on  the  criterion 
measures  or  training  school  performance  data.  Collection  of  predictor  data 
involved  giving  selected  examinees  the  same  tests  under  standardized  conditions. 
However,  for  the  criterion  data,  for  each  specialty,  the  types  of  school 
performance  data  vary,  as  well  as  the  data’s  usefulness  as  a  criterion  measure. 
For  this  reason,  under  Contract  N66001-87-D-0010  delivery  orders  7J13  and  7J17, 
we  conducted  criterion  development.  We  analyzed  a  sample  of  available  school 
performance  data,  giving  special  attention  to  identifying  any  types  of  measures 
that  tend  to  correlate  highly  with  on-the-job  performance,,  such  as  simulator  or 
hands-on  training  in  the  laboratory.  Then  we  combined  measures  that  correlated 
highly  with  each  other  into  summary  scores,  determined  reliabilities  of  the 
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criteria,  and  delivered  the  findings  in  the  form  of  technical  reports.  However, 
school  curricula  changed  over  the  course  of  data  collection,  and  the  Government 
needed  further  information  to  conduct  the  analyses  and  to  justify  the  approach 
in  joint-service  reviews. 

This  report  documents  all  procedures  and  analyses,  and  it  provides  the 
scientific  justification  for  the  criterion  measures.  We  examined  the  following 
13  Navy  schools:  Air  Traffic  Controller  (AC),  Aviation  Electrician’s  Mate  (AE), 
Aviation  Ordnanceman  (AO),  Aviation  Structural  Mechanic  -  Structures  (AMS), 
Avionics  Technician  (AV),  Electrician’s  Mate  (EM),  Electronics  Technician, 
Advanced  Electronics  Field  (ET,  AEF),  Engineman  (EN),  Fire  Controlman  (FC), 
Gunner’s  Mate  -  Phase  1  (GMG),  Machinist’s  Mate  (MM),  Operations  Specialist  (OS), 
and  Radioman  (RM).  We  also  examined  three  Army  schools:  Heavy  Anti  armor  Weapons 
Crewman  (IIH),  Field  Artillery  Fire  Support  Specialist  (13F),  and  Tank  Crewman 
(19K);  and  two  Air  Force  schools:  Air  Traffic  Control  Operator  (27230),  and 
Personnel  Specialist  (73230). 
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APPROACH 


Site  Visits 

As  part  of  an  earlier  study,  Ml  researchers  conducted  site  visits  at  the 
various  schools  (Kieckhaefer,  Bebb,  Crowe,  Corpe,  i  Zultner,  1990;  Kieckhaefer 
J  Brantner,  1990).  In  these  visits  and  in  subsequent  follow-up  questions, 
researchers  focused  on:  (a)  identifying  as  much  psychometric  information  about 
existing  tests  (including  and  especially  test  reliability  information),  (b) 
identifying  as  many  performance  or  practical  tests  as  possible,  (c)  formalizing 
informal  grading  and  grade-recording  systems  whenever  possible,  and  (d) 
determining  the  existence  or  extent  of  criterion  contamination  due  to  instructor 

knowledge  of  trainee  predictor  scores. 

RGI  researchers  toured  the  training  facilities  which  focused  on  the 
laboratories,  simulators,  and  other  hands-on  performance-measurement  situations. 
The  researchers  used  these  tours  to:  (a)  gain  an  understanding  of  the  kinds  of 
skills  required  to  perform  successfully,  (b)  gain  an  understanding  of  the 
measurement  techniques  implemented  by  the  school,  and  (c)  identify  any  new 

performance  measures  to  include  in  the  study. 

The  current  study  examined  the  18  schools  listed  in  Table  1. 
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Table  1 

Navy.  Armv-  and  Air  Force  Schools  and  Their  Abbreviations 


Service 

School 

Abbreviation 

Navy 

Air  Traffic  Controller 

AC 

Aviation  Electrician’s  Mate 

AE 

Aviation  Structural  Mechanic  -  Structures 

AMS 

Aviation  Ordnanceman 

AO 

Avionics  Technician 

AV 

Electrician’s  Mate 

EM 

Electronics  Technician,  Advanced  Electronics  Field 

ET 

Engineman 

EN 

Fire  Controlman 

FC 

Gunner’s  Mate  -  Phase  I 

GMG 

Machinist’s  Mate 

MM 

Operations  Specialist 

OS 

Radioman 

RM 

Army 

Heavy  Anti  armor  Weapons  Crewman 

IIH 

Field  Artillery  Fire  Support  Specialist 

13F 

Tank  Crewman 

19K 

Air  Force 

Air  Traffic  Control  Operator 

27230 

Personnel  Specialist 

73230 

1 
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Data  Analyses 

Under  Delivery  Order  7003,  RGI  entered  the  data  for  each  case  from  all 
schools  into  a  database  (either  from  ISS  tape  or  by  manual  data  entry).  We  then 
implemented  the  same  approach  to  data  analysis  for  each  data  set.  We  used  SPSSx 
as  the  statistical  package  software  throughout  the  data  analysis  process.  As  the 
first  step,  we  conducted  descriptive  data  analyses  using  the  SPSSx  FREQUENCIES 
procedure  to  obtain  univariate  statistics  (mean,  standard  deviation,  skewness, 
kurtosis,  minimum,  and  maximum)  on  each  variable.  We  eliminated  variables  which 
had  little  or  no  variance  and  also  variables  which  had  substantial  missing  data. 

During  the  next  step,  we  conducted  factor  analysis.  The  researchers 
excluded  from  the  factor  analysis  variables  that  consisted  of  linear  combinations 
of  other  variables.  For  example,  final  school  grade  (FSG)  consists  of  weighted 
test  scores,  and  the  researchers  did  not  include  FSG  in  any  factor  analyses. 
Then,  we  implemented  a  modified  principal  factoring  procedure  (principal  axis 
factor  analysis)  such  that  we  replaced  the  main  diagonal  elements  of  the 
correlation  matrix  with  initial  commonality  estimates  given  by  squared  multiple 
correlations. 

Next,  we  ran  a  scree  test  to  determine  how  many  factors  to  extract 
(Cattell,  1966).  We  extracted  only  that  number  of  factors  identified  by  the 
scree  test,  provided  that  the  resulting  solution  gave  interpretable  factors. 
Generally,  we  eliminated  doublets,  factors  defined  by  only  two  variables  (Harman, 
1976,  p.  129).  The  researchers  followed  this  analysis  with  a  varimax  rotation. 
Finally,  we  computed  and  saved  factor  scores  for  each  rotated  factor. 

Next,  we  computed  several  other  variables.  If  training  curricula  had 
phases  and  factor  analysis  showed  only  one  factor,  researchers  computed  phase 
variables.  If  training  had  both  written  and  performance  tests,  we  computed 
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ssparatG  variables  for  each.  If  two  or  more  factors  resulted  from  factor 
analysis,  then  we  computed  simple  sums  of  appropriate  variables  in  addition  to 

the  factor  scores. 

At  this  point,  we  took  steps  to  identify  the  fewest  number  of 
unidimensional  criterion  variables  which  would  account  for  the  criterion  variance 
in  each  school.  While  criterion  unidimensionality  may  lead  to  the  inclusion  of 
more  criterion  variables,  the  researchers  considered  this  quality  important  in 
facilitating  interpretation  of  results.  With  a  multidimensional  criterion, 
incremental  validity  demonstrated  by  one  predictor  may  result  from  only  one  of 
the  unidentified  dimensions  which  make  up  the  criterion.  With  a  unidimensional 
criterion,  the  incremental  validity  demonstrated  by  one  predictor  battery  over 
another  shows  clear  evidence  of  superiority. 

As  the  final  step  in  identifying  criterion  variables,  researchers  obtained 
a  correlation  matrix  of  all  factor  scores,  composite  variables,  and  the  official 
final  school  grade.  With  the  objective  of  selecting  the  fewest  unidimensional 
criteria,  we  used  the  evidence  provided  by  these  data  to  recommend  criterion 
variables  to  include  in  the  validity  studies.  We  then  obtained  descriptive 
statistics  on  the  recommended  criterion  variables. 

As  the  final  step  in  data  analysis,  researchers  developed  estimates  of 
criterion  reliability.  The  recommended  criterion  variables  constitute 
combinations  (or  composites)  of  the  tests  administered  in  training.  Hence, 
researchers  developed  reliability  estimates  of  each  component  test  first,  and 
then  combined  those  estimates  appropriately  to  obtain  a  reliability  estimate  for 
the  composite  criterion  variable.  See  Appendix  A  for  detailed  computational 
procedures  for  finding  composite  reliability  and  an  illustrative  example. 

In  all  cases  where  researchers  collected  data,  the  communality  estimates 
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provided  by  the  factor  analyses  served  as  an  estimate  of  reliability.  For  those 
few  schools  supported  by  the  ISS  system  in  Millington,  TN.  researchers  could  also 
obtain  Kuder-Richardson  estimates  as  a  second  estimate  of  test  reliability.  That 
system  used  the  KR-20  formula  to  provide  estimates  for  each  version  of  the  tests 
used  in  the  school.  In  these  cases,  researchers  determined  the  median  KR 
estimate  for  each  test  and  proceeded  to  combine  reliability  estimates  for  the 
composite.  In  these  few  cases,  this  provided  a  second  (and  generally  higher) 
reliability  estimate  for  the  composite  criterion  variable.  The  researchers  did 

not  corrGct  tgI iabil itiGS  for  rangG  rostriction. 

Statisticians  consider  the  factor  analytic  procedure  described  above 
exploratory  factor  analysis.  The  researchers  considered  but  did  not  do 
confirmatory  factor  analysis  for  two  primary  reasons.  First,  while  previous 
researchers  (Kieckhaefer  et  al.,  1990;  Kieckhaefer  &  Brantner,  1990)  did  some 
work  in  this  area,  the  earlier  studies  used  small  sample  sizes.  Without  a  firm 
theoretical  or  empirical  rationale  for  postulating  a  given  factor  structure  and 
without  specific  hypotheses  to  test,  the  present  researchers  thus  conducted 
exploratory  factor  analysis.  We  selected  principal  axis  factor  analysis  for  its 
variance  maximizing  properties  and  its  specification  of  the  relationship  of 
commonality  to  reliability.  The  second  reason  for  exploratory  factor  analysis 
concerns  the  substantial  curriculum  changes  that  took  place  in  many  of  the 
schools.  These  changes  argued  against  any  a  priori  expectation  of  a  specified 
factor  structure.  To  maintain  consistency  in  their  approach  across  all  schools, 
even  ones  which  had  not  changed,  the  researchers  implemented  the  same  procedure 

in  all  cases. 

In  schools  which  had  a  change  in  curriculum  or  testing  plan,  we  treated  the 
two  or  more  curricula  separately.  We  conducted  multivariate  analysis  of  variance 
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(MANOVA)  on  the  criterion  variables  to  assess  differences  in  means  and  covariance 
matrices  between  curricula.  We  also  conducted  multivariate  analysis  of 
covariance  (MANCOVA)  on  the  criterion  variables  controlling  for  ability  as 
assessed  by  pre-enlistment  ASVAB  scores. 

To  further  evaluate  differences  between  versions,  we  computed  effect  sizes 
as  measures  of  the  practical  significance  of  the  differences  between  the 
unadjusted  means  across  curricula.  We  used  the  formula: 

o 

where  M,  is  the  mean  of  Group  A  on  the  variable  of  interest,  Mj  is  the  mean  of 
Group  B,  and  a  is  the  estimated  population  standard  deviation  (Cohen  &  Cohen, 
1977).  Cohen  and  Cohen  (1977)  say  that  .2  represents  a  small  effect  size,  .5  is 
medium,  and  .8  is  large.  Thus,  while  the  MANOVA  tests  may  indicate  that 
curricula  differ  from  one  another  in  terms  of  statistical  significance,  the 
effect  sizes  represent  the  degree  of  the  differences  in  standard  deviation  units. 

Finally,  we  also  compared  factor  solutions  from  two  or  more  curricula  using 
the  coefficient  of  congruence  (Cattell,  1978;  Gorsuch,  1983)  to  assess  the 
resemblance  of  factors  from  one  curriculum  to  those  from  another. 
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FINDINGS 


The  data  collection  and  analyses  described  in  the  Approach  section  resulted 
in  the  findings  and  recommended  criterion  variables  described  in  this  section. 
First,  the  researchers  describe  the  tests  used  in  the  various  Navy,  Air  Force, 
and  Army  schools.  As  a  minimum,  they  recommended  including  the  FSG  for  each 
school.  This  recommendation  stems  from  political  reasons  as  well  as  because 
previous  validity  studies  tended  to  use  at  least  (if  not  only)  FSG. 

The  subsections  below  present  RGI’s  recommended  criterion  variables  for 
each  school.  As  noted  above,  RGI  estimated  reliability  from  the  communal ity 
estimates  in  the  factor  analyses.  Appendix  A  contains  the  detailed  computational 
procedures  for  finding  composite  reliability  and  an  illustrative  example. 
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Navy  Schools 

Air  Traffic  Controller  (AC) 

Descriptions  of  Variables.  The  Navy  Air  Traffic  Control  (AC)  School 
consists  of  three  blocks;  I.  Base  Operations,  II.  Tower,  and  III.  Radar.  Table 
2  below  lists  the  tests  which  make  up  each  Block  and  the  weights  which  the  school 
assigns  to  each  test  in  calculating  the  Final  School  Grade  (FSG).  The  written 
tests  have  a  (W)  following  their  description,  and  the  performance  (laboratory) 
tests  have  a  (P)  following  their  description.  The  school  scores  all  these  tests 
and  FSG  on  a  scale  from  0  to  100. 

The  school  defines  FSG  as  the  weighted  sum  (composite)  of  the  individual 
tests.  Percentages  representing  the  weights  applied  by  the  school  to  each  test 
score  in  composing  the  FSG  follow  the  test  descriptions  in  Table  2.  Students 
having  initial  test  scores  below  70  retake  the  test.  For  students  who  pass  the 
first  or  subsequent  retest,  the  school  records  scores  of  70  and  uses  these  scores 
in  calculating  the  FSG.  R6I  entered  the  initial  test  scores  in  the  database. 
Therefore,  an  FSG  which  a  researcher  might  calculate  using  the  initial  scores 
differs  from  the  FSG  calculated  by  the  school  using  the  retest  scores. 

The  Student  action  cooe/Status  variable  describes  the  category  of  student 
outcome  (e.g.,  graduated,  disenrolled;  see  Appendix  B) .  The  class  convening  date 
gives  the  year,  month,  and  day  the  class  started.  The  Block  II  (Tower)  Airman’s 
Written  Test  constitutes  a  Federal  Aviation  Administration  (FAA)  secure  test. 
In  Block  III,  students  take  an  Identification  &  Vectoring  Procedures  performance 
test  in  a  radar  lab  simulator.  Instructors  grade  students  on  a  Satisfactory 
("S")  or  Unsatisfactory  ("U")  basis.  Students  who  fail  undertake  remedial 
training,  and  reportedly  all  students  pass  the  retest.  RGI  entered  only  the 
initial  scores. 
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Tabic  2 

Descnptions  of  AC  \'ariablcs 


Block 

Test® 

Weight 

I 

Air  Traffic  Control  Fundamentals  (W') 

5% 

1 

Airspace  &  Time  (WO 

5% 

I 

Navigational  Aids  (W) 

5% 

I 

Charts  &  Publications  (WO 

5% 

1 

Right  Plans  ( WO 

5% 

I 

Aviation  W'eather  (WO 

Block  1  subtotal  30% 

5% 

11 

Basic  Knowledge  (WO 

5% 

II 

General  Control  &  Equipment  (W) 

5% 

!I 

Visual  Right  Rules  Procedures  (WO 

5% 

II 

Instrument  Right  Rules  Procedures  (WO 

5% 

11 

Airman  s  Written  Test  Cenification  (W) 

10% 

11 

Local.  Ground.  Right  Data  (P) 

Block  II  subtotal  40% 

10% 

III 

Identification  &  Vectonng  Procedures  (W) 

5% 

III 

Identification  &  Vectonng  Procedures  (P) 

0% 

III 

Air  Surveillance  Radar  Procedures  (WO 

2J% 

III 

Air  Surv'eiilance  Radar  Procedures  (P) 

5% 

III 

Precision  Approach  Radar  Procedures  (WO 

25% 

III 

Precision  Approach  Radar  Procedures  (P) 

5% 

III 

Arrival  Procedures  (WO 

5% 

III 

AnTr-al  Procedures  (P) 

Block  III  subtotal  30% 

Final  Schcwl  Grade  100% 

5% 

*Key  10  icticrs  in  parentheses:  \V  =  Written  tests.  P  =  Performance  or  laboratory  tests 


The  school  provided  information  on  students’  rank  and  the  number  of 
students  in  the  class,  each  of  which  could  range  from  1  to  30.  We  computed  a 
measure  of  class  standing  using  student  rank  (r)  and  class  size  (n):  (class  size 
-  rank) /class  size  =  (n-r)/n.  The  class  standing  variable  puts  students  on  a 
metric  of  0  to  approaching  1,  so  that  the  higher  the  standing,  the  better  the 
student  did  relative  to  the  other  members  of  the  class.  Further,  this  measure 
takes  into  account  class  size.  For  example,  a  person  who  ranked  first  out  of  a 
class  of  30  would  have  a  standing  of  .97,  while  a  person  who  ranked  first  out  of 
a  class  of  5  would  have  a  standing  of  .80. 

Sample.  We  dropped  cases  which  had  missing  data  on  any  of  the  test 
variables.  This  resulted  in  a  total  sample  size  of  N=81  out  of  an  initial  sample 
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of  N=lll.  Then,  we  analyzed  the  20  variables  shown  in  Table  3. 

Results  and  Discussion.  Table  3  displays  descriptive  statistics  on  the  AC 
tests.  Factor  analysis  indicates  three  factors.  Due  to  the  small  sample  size, 
we  present  these  results  as  suggestive  and  not  definitive.  Table  4  shows  the 
communal ities,  factor  loadings,  and  factor  score  coefficients.  The  primary 
factor  consists  of  the  sum  of  all  the  written  tests,  except  the  Precision 
Approach  Radar  Procedures  written  test  which  loads  more  heavily  on  the  third 
factor.  Thus,  the  major  factor  represents  the  knowledge  the  student  has 
regarding  the  subject  matter  of  Navy  Air  Traffic  Control  covering  base 
operations,  towers,  and  radar. 


Table  3 

Descriptive  Statistics  for  AC  Variables 


Test* 

Mean 

Std  Dev 

Skew 

Kurtosis 

Minimum 

Maximum 

Air  Traffic  Control  Fund.  (W) 

87.315 

7.713 

-.342 

-.632 

70.000 

100.000 

Airspace  &.  Time  (W) 

82.975 

9.624 

-.246 

-.395 

55.000 

100.000 

Navigational  Aids 

83.352 

9.919 

-.243 

-.710 

58.000 

100.000 

Charts  &  Publications  (W) 

84.226 

8.033 

-.398 

.(M8 

58,000 

98.000 

Flight  Plans  (W) 

86.278 

8.582 

-.765 

.928 

57300 

100.000 

Aviation  Weather  (W) 

81.737 

8-584 

-.333 

-385 

62.000 

97.100 

Basic  Knowledge  (W) 

80.290 

10.512 

-.484 

-.046 

51.100 

100.000 

General  Control  &  Equipment  (W) 

80.226 

7.905 

-.248 

.418 

58.000 

100.000 

Visual  Flight  Rules  Procedures  (W) 

78.986 

9.235 

-.035 

-.461 

60.000 

98.800 

Instrument  Flight  Rules  Proc.  (W) 

79.037 

8.815 

-.015 

-.718 

60.000 

96.000 

Airman's  Written  Test  Cert,  (W) 

79.000 

6.243 

-.098 

.621 

60.000 

96.000 

Local,  Ground,  Flight  Data  (P) 

89.725 

5.487 

-.409 

-.071 

73300 

100.000 

Went.  &  Vectoring  Proc.  (W) 

82.432 

8340 

-.128 

-.222 

60.000 

100.000 

Went.  &  Vectoring  Proc.  (P) 

.877 

.331 

-2.333 

3328 

.000 

1.000 

Air  Surveillance  Radar  Proc,  (W) 

85.864 

9313 

-389 

-.225 

60.000 

100.000 

Air  Surveillance  Radar  Proc.  (P) 

90.654 

8.469 

-.993 

.810 

60.000 

100.000 

Precision  Approach  Radar  Proc.  (W) 

85,123 

9.649 

-.603 

.294 

55.000 

100.000 

Precision  Approach  Radar  Proc.  (P) 

93.043 

7.285 

-2,306 

7,355 

58.000 

100.000 

Arrival  Procedures  (W) 

86.481 

7.744 

-339 

-.169 

65.000 

100.000 

Arrival  Procedures  (P) 

86.284 

14.157 

-3.161 

16.405 

.000 

100.000 

Note.  N  =  81. 

®Kcy  to  letters  in  parentheses:  W  =  Written  tests,  P  =  Performance  or  laboratory  tests 
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Tabic  4 

Communalities.  Factor  Loadings  and  Factor  Score  Coefficients  tor  AC  Variables 


\’dnabic 


Communamv 


Factor  Loadings 
1  2  3 


Factor  Score  Cocificicnis 
1  2  3 


Air  Traffic  Control  Fund.  (W) 

.528 

.714 

Airspace  &  Time  (W) 

.604 

.776 

Navigational  Aids  (V»T 

JS21 

.717 

Charts  &  Publications  (W) 

i80 

.686 

Flight  Plans  ("W*) 

.456 

.619 

Aviation  Weather  (W) 

.502 

.705 

Basic  Knowledge 

i96 

.743 

General  Control  &  ^uipmeni  (W) 

.465 

.675 

Visual  Flight  Rules  Procedures  (W) 

.568 

.723 

Instrument  Flight  Rules  Proc.  (W) 

i89 

.727 

Airman’s  Written  Test  Cert.  (V/) 

.403 

.620 

Local.  Ground.  Flight  Data  (P) 

.546 

.263 

Idem.  &  Vectoring  Proc.  (W^) 

.181 

.353 

Idem.  &  V'ectonng  Proc.  (P) 

.043 

.029 

Air  Surveillance  Radar  Proc.  (W) 

.217 

.327 

.\  urveiilance  Radar  Proc.  (P) 

.356 

.023 

P:  iion  Approach  Radar  Proc.  (W) 

.326 

.335 

Precision  Approach  Radar  Proc.  (P) 

.161 

-.090 

Arrival  Procedures  (W) 

.372 

563 

.Aniv'al  Procedures  (P) 

.243 

.129 

-.133 

-.004 

.104 

-.193 

-.064 

.024 

-.034 

.180 

-.029 

-.013 

-.007 

-.no 

.119 

-.020 

-.146 

-.023 

-.330 

.135 

-.126 

-.295 

-.072 

.260 

.067 

-.079 

.137 

-.004 

.076 

.102 

-.084 

.048 

.093 

.187 

.132 

.026 

.138 

.054 

.075 

.074 

-.003 

.021 

.200 

.078 

.108 

.091 

.031 

.204 

.136 

.147 

.128 

.124 

.105 

.083 

.060 

.008 

.017 

.370 

-583 

.029 

.264 

-.490 

.202 

.127 

.024 

.065 

.080 

.009 

.205 

.005 

.023 

.075 

.276 

.184 

.002 

.110 

.059 

.596 

-.028 

-.052 

.366 

-.027 

.189 

.422 

.008 

.122 

.224 

.385 

-.068 

-.019 

.157 

-.020 

.235 

.012 

.052 

.071 

.024 

.469 

.083 

-.011 

.199 

.098 

The  second  factor  consists  of  the  performance  tests,  except  the 
Identification  and  Vectoring  Procedures  performance  test,  which  loads  on  none  of 
the  factors  and  has  a  low  communal ity  (.043).  The  Local,  Ground,  and  Flight  Data 
performance  test  also  loads  more  on  Factor  3  than  Factor  2.  Thus,  the  second 
factor  represents  a  skills  dimension,  assessed  by  student  performance  on  the  lab 

tests. 

The  third  factor  contrasts  the  Precision  Approach  Radar  Procedures  “written 
test  with  the  Charts  and  Publications  written  test  and  the  Local,  Ground,  and 
Flight  Data  performance  test.  Factor  3  highlights  the  discrepancy  between 
written  knowledge  of  using  radar  for  aircraft  approaches  versus  written  knowledge 
of  charts  and  publications  and  performance  in  the  Local,  Ground,  and  Flight  Data 
labs. 
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To  clarify  some  of  the  ambiguity  in  the  above  factors,  a  forced  two-factor 
solution  yielded  the  written  tests  defining  the  first  factor  and  the  performance 
tests  (except  Identification  and  Vectoring  Procedures)  defining  the  second. 
However,  the  loading  of  the  Local,  Ground,  and  Flight  Data  lab  on  the  second 
factor  of  this  two-factor  solution  was  considerably  lower  compared  to  the  three- 
factor  solution,  thus,  the  empirical  results  favor  three  factors.  The  drawback 
of  the  third  factor  concerns  its  variables  overlapping  with  Factors  1  and  2.  A 
subject  matter  expert  at  the  AC  school  could  not  interpret  this  third  factor. 
While  this  Factor  3  may  have  significance,  this  overlap  and  the  interpretabil ity 
problem  lead  the  researchers  to  defer  recommendations  of  this  factor. 

The  researchers  developed  composite  variables  to  represent  the  dimensions 
uncovered  by  the  factor  analysis.  "Knowledge"  consisted  of  the  mean  of  all  15 
written  tests.  "Skills"  consisted  of  the  mean  of  the  four  performance  tests 
(except  the  pass-fail  Identification  and  Vectoring  Procedures  lab).  "Contrast" 
consisted  of  (the  Precision  Approach  Radar  Procedure  written  test  -  Charts  and 
Publications  -  Local,  Ground,  and  Flight  Data)/3.  "Block  I"  consisted  of  the 
mean  of  all  six  Block  I  (Base  Operations)  tests.  "Block  11"  consisted  of  the 
mean  of  all  six  Block  II  (Tower)  tests.  "Block  III"  consisted  of  the  mean  of  all 
seven  Block  III  (Radar)  tests.  Table  5  lists  descriptive  statistics  on  the 
composite  variables. 

Next,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  6  displays  the  intercorrelations  among  the  composite  variables 
and  the  factors.  Since  FSG  and  FSG2  correlate  .96,  this  offers  no  support  for 
using  initial  test  scores  to  compute  FSG. 

The  Knowledge  composite  correlates  .98  with  Factor  1,  while  the  Skills 
composite  correlates  .88  with  Factor  2.  The  Knowledge  and  Skills  composites 
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Table  5 

StniKiics  for  AC  Composite  \  anabies 


Composite 


Rcliabilitv'^ 

.Mean 

Sid  Dev 

Skew 

Kunosis 

Minimum 

Maximur 

.91 

82.922 

5.733 

.398 

-.655 

71.413 

98.473 

.59 

89.926 

5.861 

.1.779 

6.833 

60i00 

99500 

.59 

.29.609 

4.898 

.431 

ilO 

-39567 

-13.767 

S/A 

84.701 

4.579 

.396 

-.519 

74530 

97.450 

.90 

84.338 

4.764 

.422 

-.606 

74,780 

97.450 

.87 

W.397 

6.697 

.180 

-.713 

69.767 

99.183 

.84 

81.211 

5.889 

.373 

-.728 

71.250 

97.033 

.63 

87.126 

5.074 

-.866 

2.406 

66.000 

97.214 

S/A 

.441 

.278 

.173 

-1.177 

.000 

.930 

Knowledge” 
Skills' 
Contrast^ 
FSG 
FS02* 
Block  I* 
Block  U* 
Block  Ill*’ 
Standing’ 


Vntr-  The  rcsearchere  did  not  tnclude  the  Block  III  Identification  &  Vectoring  Procedures  periormanee  test  in  any  of  the  abwe 

rj  ,h, «.... .( .h. . . . . . .  «• 

“These  are  composites  based  on  communalit>-  estimates  and  the  procedures  desenbed  m  the  text. 

‘’Know'ledge;  Mean  of  all  lAiritten  Tests 
'Skills:  .Mean  of  all  Performance  Tests 
“Contrast:  (P.ARPW  -  CP  -  LGFDV3 
®FSG2:  FSG  computed  using  initial  test  scores 
-Block  I:  Mean  of  Block  I  fBase  Operations)  Tests 
SBlock  11:  Mean  of  Block  II  (Tower)  Tests 
"Block  111:  Mean  of  Block  111  fRadar)  Tests 
’Standing:  Standing  =  (class  size  -  rankl/dass  size 


Table  6 

Correlations  Among  AC  Composites  and  Factors 


Composuc 

Factor  1 

Factor  2 

Factor  3 

Knowledge  Skills 

Contrast 

FSG 

FSG(2) 

Block  I 

Block  11 

Block  111 

Factor  2 

.05 

Factor  3 

.02 

-.04 

Knowledge® 

.98 

.18 

.16 

.20 

SkilU** 

.12 

.88 

-.15 

-.12 

Contrast® 

-.26 

-.00 

.85 

-.13 

-.31 

FSG 

.91 

.36 

-.00 

.94 

.39 

.96 

FSG2^ 

.94 

.38 

.03 

.96 

.44 

-.22 

.87 

Block  I® 

.96 

-.05 

-.04 

.92 

.10 

-.30 

.85 

,80 

Block  II  ^ 

.93 

.28 

.06 

.93 

.26 

-.21 

.90 

.94 

53 

.83 

. 

Block  lIlS 
Standing^ 

.44 

.84 

.79 

.36 

.26 

,04 

59 

.88 

.77 

,40 

.16 

-.22 

.67 

.90 

.70 

.90 

.36 

.80 

.69 

Note:  The  researchers  did  not  include  the  Block  111  Identirtcation  &  Vectoring  Procedures  performance  test  in  any  of  the  above 
composites. 

“Knowledge;  Mean  of  all  Written  Tests 
^Skills:  Mean  of  all  Performance  Tests 
'Contrast:  (PARPW  -  CP  -  LGFD)/3 
**1^02:  FSG  computed  using  initial  test  scores 
“Block  1:  Mean  of  Block  I  (Base  Operations)  Tests 
^Block  II:  Mean  of  Block  II  (Tower)  Tests 
*Block  III:  Mean  of  Block  III  (Radar)  Tests 
^’Standing:  Standing  =  (class  size  •  rank)/class  size 
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assess  relatively  independent  aspects  of  Air  Traffic  Controller  school 
performance  (r  =  .20).  FSG  correlates  .94  with  Factor  1  and  only  .38  with  Factor 
2.  Knowledge  correlates  .96  with  FSG,  while  Skills  only  correlates  .44  with  FSG. 
Thus,  FSG  largely  reflects  written  test  scores,  as  indicated  by  the  preponderance 
of  such  tests  in  the  AC  school. 

The  measure  of  Base  Operations  (Block  I)  correlates  most  closely  with 
Factor  1  (r  =  .96),  as  does  the  Tower  (Block  II)  measure  (r  *  .93).  The  Radar 

measure  (Block  III)  correlates  most  closely  with  Factor  2  (r  *  .79).  The 

) 

correlations  between  these  block  measures  and  the  Knowledge  and  Skills  composites 
also  reflect  these  patterns.  Thus,  Blocks  I  and  II  primarily  measure  written 
test  ability  and  Block  III  primarily  assesses  lab  performance.  Finally,  Standing 
correlates  very  highly  with  Factor  1  (.84),  Knowledge  (.88),  and  FSG  (.90),  which 
again  reflects  the  prevalence  of  written  test  scores  in  this  school. 

Recommendations.  In  addition  to  FSG,  use  the  mean  of  the  four  performance 
tests,  excluding  the  pass-fail  Identification  and  Vectoring  Procedures  lab  test. 
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Aviation  Electrician’s  Mate  (AE) 

Description  of  Variables.  The  Aviation  Electrician’s  Mate  (AE)  school 
divides  the  curriculum  and  testing  sequence  into  15  units.  The  school  assesses 
student  progress  using  Knowledge,  Performance,  and  Comprehensive  tests.  The 
tests  have  a  valid  range  from  0-100.  Table  7  presents  the  units  and 
corresponding  tests  for  the  AE  curriculum.  The  school  sets  the  minimum  passing 
score  for  each  test  at  63.  Students  who  do  not  achieve  the  minimum  passing  score 
take  one  retest  per  test.  For  students  passing  the  retest,  the  AE  school  assigns 
a  score  of  63  which  counts  toward  the  student’s  final  school  grade  (FSG).  For 
students  who  fail  the  retest,  an  Academic  Review  Board  forms  a  recommendation  as 
to  the  student’s  status.  For  the  purposes  of  this  study,  RGI  entered  all  initial 
test  scores  into  the  database. 

For  each  student,  the  school  computes  a  unit  grade  from  the  appropriate 
Knowledge,  Performance,  and  Comprehensive  test  scores.  The  school  then 
calculates  the  FSG  as  a  weighted  average  of  the  unit  grades.  See  Appendix  C  for 
test  and  unit  weights.  The  Student  Action  Code  variable  represents  the 
graduation  status  of  the  student.  See  Appendix  B  for  descriptions  of  the  various 
disenrollment  categories. 

Sample.  The  sample  consisted  of  231  students.  We  excluded  from  some 
analyses  the  29  cases  that  did  not  take  every  test. 

Results  and  Discussion.  We  computed  descriptive  statistics  for  each  of  the 
test  variables.  Table  8  lists  the  descriptive  statistics  for  all  variables. 

Next,  we  conducted  a  factor  analysis  on  the  set  of  all  test  variables.  The 
scree  test  and  the  factor  loadings  indicated  two  factors.  Table  9  lists  the 
commonalities,  factor  loadings,  and  factor  score  coefficients  for  each  of  the 
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Tabic  7 


AF  School  Instructional  Units  and  Tcf^ts 


Unit  Test  No.  Dcscnption  (Name  -  T>pc  of  Tcst^) 


0 

301 

1 

311 

322 

321 

3 

332 

331 

4 

342 

341 

5 

352 

351 

6 

361 

- 

372 

371 

S 

3S2 

381 

9 

392 

393 

394 

395 

396 

391 

10 

402 

403 

404 

401 

11 

412 

411 

12 

422 

421 

13 

432 

433 

434 

435 

436 

431 

14 

442 

441 

15 

451 

Knowledge  test  covcnng  Whole  Numbers.  Fractions,  Decimals,  and  Basic  Algebra  (Math  -  K) 

Knowledge  test  covering  Basic  DC  Theorv'  (DC  Theory- K) 

Performance  test  covering  DC  Series-Parallel  Circuits  (DC  Circuits-P) 

Knowledge  test  covering  Basic  DC  Circuits  (DC  Circuiis-K) 

Performance  test  covering  Basic  AC  Theory  (AC  Thcory-P) 

Knowledge  test  covering  Basic  AC  Theory  (AC  Theory-K) 

Performance  test  covering  LC  Parallel  Resonant  Circuits  (LC  Circuits-P) 

Comprehensive  knowledge  test  covering  all  subject  material  on  tests  #301  through  #342  (Comp.  1) 

Performance  test  covering  Diodes.  Transistor  Amplifiers.  Integrated  Circuits,  and  Cathode  Ray  Tubes  (Diodes,  ctc.-P) 
Knowledge  test  covering  Applications  of  Electronic  Circuits  (Elect  Applic.-K) 

Knowledge  test  covenng  Digital  Fundamentals  (Digital-K) 

Performance  test  covering  Maintenance  Data  Forms,  an  Intro  to  Troubleshooting,  and  Intcnor  Light  (Maim.  ctc.-P) 
Knowledge  test  covering  Basic  Troubleshooting  (Troubleshooting-K) 

Performance  test  covenng  AC  and  DC  Power  Distribution  Systems  (AC/DC  Power-P) 

Knowledge  test  covenng  Generators  (Gcncraiors-K) 

Performance  test  covering  Hydraulics  Arresting  Gear  System  (Hydr.  Gcar-P) 

Performance  test  covering  Hydraulics  Speed  Brake  System  (Hydr.  Brake-P) 

Performance  test  covering  Hydraulics  Flap  System  (Hydr.  Flap-P) 

Performance  test  covering  the  Hydraulics  Landing  Gear  System  (Hydr.  Landing-P) 

Performance  test  covering  the  Hydraulics  Nosewheel  Steering  System  (Hydr.  Steering-P) 

Comprehensive  knowledge  test  covering  ail  subject  material  on  tests  #352  through  #396  (Comp  2) 

Performance  test  covering  the  Fuel  Quantity  System  (Fuel  System-P) 

Performance  test  covering  the  Simple  Synchros  System  (Synchros-P) 

Performance  test  covering  the  Pitot  Static  System  (Pitot  Static-P) 

Knowledge  test  covering  Indicating  Systems  (Indicating-K) 

Performance  test  covering  Attitude  Headings  and  Reference  Systems  (Attitude  Hdgs.-P) 

Knowledge  test  covering  the  Navigation  System  (Navigation-P) 

Performance  test  covering  the  Inertial  Navigation  System  (Inertial  Nav.-P) 

Knowledge  test  covering  the  Inertial  Navigation  and  Automatic  Flight  Control  System  (Inenial  Nav.-K) 

Performance  test  covering  the  Jet  Ignition  System  (Jet  Ignition-P) 

Performance  test  covering  the  Engine  RPM  Indicating  System  (Engine  RPM-P) 

Performance  test  covering  the  Turbine  Inlet  Temperature  System  (Turbine  Temp.-P) 

Performance  test  covering  Anti-Ice  and  Deice  Systems  (Ice  Systems-P) 

Performance  test  covering  Anti-ice  and  Deice  Systems  (Ice  Systems2-P) 

Knowledge  test  covering  Engine  instruments  (Eng  Instr.-K) 

Performance  test  covering  the  Angle  of  Attack  System,  the  Fire  Warning  System,  and  Scheduled/Unschedulcd 
Maintenance  (Attack,  ctc.-P) 

Comprehensive  test  covering  all  subject  material  on  tests  #402  through  #442  (Final  Comp.) 

Knowledge  test  covering  Avionics  Corrosion  (Avionics-K) 


®Typc  of  test  refers  to  either  Knowledge  (K)  or  Performance  (P) 


18 


Table  8 


Descnptn-e  Stansucs  for  AH  Vanahjes 


Vanablc 

J1 

.Mean 

Sid  Dev 

Skew 

Kurtosts 

Minimum  Maximum 

Math  •  K 

228 

&4.890 

11.624 

-1.635 

3.311 

3230 

100.00 

DC  Thcorv-  -  K 

228 

81.271 

12,824 

..673 

.314 

30.30 

100.00 

DC  Circuits  -  P 

226 

91.692 

13.319 

-2.135 

3.929 

50.00 

100.00 

DC  Circuits  -  K 

226 

80.147 

14.996 

.  .981 

385 

26.90 

100.00 

AC  Thcorv  -  P 

222 

98.712 

4.043 

4.211 

21.940 

68.00 

100.00 

AC  Thcorv*  -  K 

222 

82.744 

11.667 

■  .746 

.239 

38.70 

100.00 

LC  Circuits  -  P 

220 

97.350 

5.751 

-3.693 

21.840 

50.00 

100.00 

Comp.  1 

220 

78.717 

11.496 

■  .488 

.068 

4130 

100.00 

Diodes,  etc.  -  P 

218 

95.477 

7.822 

-2.975 

11.607 

50.00 

100.00 

Elect  Applic.  -  K 

218 

77.CM5 

12.232 

-  .563 

.199 

34.30 

100.00 

D  *al  -  K 

216 

85-590 

9.559 

-1.198 

2.689 

3730 

100.00 

Maim.,  etc.  -  P 

215 

91.484 

13.694 

-2.157 

.166 

50.00 

100.00 

Troubleshtg  -  K 

215 

83.907 

10.401 

-  .615 

-  .171 

52.00 

100,00 

AC/DC  Power  -  P 

213 

86.178 

18.058 

-1.296 

.088 

50.00 

100.00 

Generators  -  K 

212 

76.566 

13.940 

-327 

-  .219 

3730 

100.00 

Hydr.  Gear  -  P 

208 

89,958 

12.202 

-1.927 

3347 

50.00 

100.00 

Hydr.  Brake  -  P 

208 

86.159 

16.595 

-1.358 

309 

50.00 

100.00 

Hydr.  Flap  -  P 

208 

91.457 

13.380 

-2.267 

.427 

50.00 

100.00 

Hydr.  Landing  •  P 

208 

91.697 

12.693 

-2.298 

4.820 

50.00 

100.00 

Hydr.  Steenng  -  P 

208 

89.212 

15.098 

-1,826 

2.145 

50,00 

100.00 

Comp.  2 

206 

76.650 

11.204 

-  .126 

-  .614 

44.00 

100.00 

Fuel  System  •  P 

206 

90.601 

15J18 

-2.994 

11.423 

00,40 

100.00 

Synchros  -  P 

206 

92.199 

10.284 

-2.348 

6300 

50.00 

100.00 

Pitot  Static  -  P 

206 

91.749 

7.312 

-369 

-.698 

70.00 

100.00 

Indicating  -  P 

206 

78,974 

10.660 

-  .425 

-.103 

48.60 

100.00 

Attitude  Hdgs.  -  P 

206 

85.949 

17.833 

-1.605 

2.270 

00.40 

100.00 

Navigation  -  P 

206 

74.598 

11.280 

-  .027 

-  .462 

47.20 

100.00 

Inenia!  Nav.  -  P 

203 

90.726 

11.157 

-3.682 

22332 

00.40 

100.00 

Inertial  Nav.  -  K 

203 

76.717 

11JS3 

-.228 

-  .462 

46.70 

100.00 

Jet  Ignition  •  P 

203 

89.063 

13.824 

-2.748 

10.015 

00.40 

100.00 

Engine  RPM  -  P 

203 

94.783 

6.238 

-2.638 

10.058 

50.00 

100.00 

Turbine  Temp.  -  P 

203 

92.013 

12376 

-4.496 

27.489 

00.40 

100.00 

Ice  Systems  -  P 

203 

92.201 

13.143 

4.288 

24.166 

00.40 

100.00 

Ice  Systems2  -  P 

203 

89.885 

15.745 

-2.288 

4.895 

9.60 

100.00 

Eng  Instr.  -  K 

203 

82.067 

10.762 

-.623 

-.131 

46.70 

100.00 

Atuck,  etc.  -  P 

203 

88.557 

10.828 

-1381 

3.041 

50.00 

100.00 

Final  Comp. 

203 

75.123 

10372 

-313 

.487 

34.00 

100.00 

Avionics  -  K 

203 

85.564 

11.062 

-1.882 

6.843 

22.20 

100.00 
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Table  9 

(■’ommunaiitics.  Factor  Loadings  and  Factor  Score  Coefficients  for  AE  Variables 


Factor  Loadings 


Factor  Score  Coefficients 


\'ariable 

Communality 

Factor  1: 
Knowlcdee 

Factor  2: 
Performance 

Factor  1: 
Knowlcdec 

Factor  2: 
l>jfannanoe 

Math  -  K 

.233 

.425 

.229 

.044 

.016 

DC  Thcon*  -  K 

.-^24 

.635 

.142 

.150 

-.079 

DC  Circuits  -  P 

.174 

.319 

.267 

.022 

.051 

DC  Circuits  -  K 

.443 

.603 

.281 

.105 

.022 

AC  Thcorr’  •  P 

.003 

.048 

.024 

-.001 

-.003 

AC  Theory  -  K 

.395 

343 

.318 

.065 

.050 

LC  Circuits  ■  P 

.141 

.372 

.047 

-.059 

-.036 

Comp.  1 

.386 

389 

.198 

.111 

-.029 

Diodes,  etc.  -  P 

.022 

.149 

-.007 

-.025 

-.021 

Elect  Apphe.  -  K 

.377 

330 

.311 

-.067 

.045 

Digital  -  K 

.413 

357 

.321 

.086 

.W5 

Maint.  etc  -  P 

.162 

.229 

.332 

-.004 

.081 

Troubleshooting  -  K 

.349 

332 

,258 

.079 

.027 

AC/DC  Power  -  P 

.193 

.236 

.371 

-.012 

.094 

Generators  •  K 

.447 

378 

.337 

.080 

.062 

Hydr.  Gear  -  P 

.181 

.111 

.411 

-.044 

.133 

Hvtlr.  Brake  -  P 

.238 

.133 

.469 

-.062 

.177 

Hydr.  Flap  -  P 

.311 

.004 

358 

-.105 

.263 

Hv'dr.  Landing  -  P 

.179 

.225 

.359 

-.014 

.090 

Hydr.  Steenng  -  P 

.192 

.020 

.438 

-.067 

.162 

Comp.  2 

.416 

381 

.281 

.092 

.010 

Fuel  System  -  P 

.216 

.141 

.444 

-.043 

.145 

Sv-nchros  -  P 

.091 

.186 

.238 

-.002 

.046 

Pitot  Static  -  P 

.201 

.159 

.420 

-.034 

.131 

Indicating  -  K 

.334 

358 

.151 

.099 

-.049 

Attitude  Hdgs.  -  P 

.013 

.100 

.056 

.003 

.012 

Navigation  -  P 

.434 

386 

.300 

.094 

.035 

Inertial  Nav,  -  P 

.048 

.053 

.214 

-.027 

.061 

Inertial  Nav.  -  K 

.390 

399 

.178 

.115 

-.040 

Jet  Ignition  -  P 

.127 

.127 

333 

-.032 

.101 

Engine  RPM  -  P 

.071 

.259 

.066 

.031 

-.015 

Turbine  Temp.  -  P 

.036 

.191 

-.001 

.023 

-.022 

Ice  Systems  -  P 

.012 

.049 

.099 

-.022 

.026 

Ice  Systems2  -  P 

.005 

-.067 

-.026 

-.005 

.005 

Eng  Instr.  -  K 

.348 

383 

.091 

.113 

-.081 

Attack,  etc.  -  P 

.003 

.002 

.061 

-.010 

.021 

Final  Comp. 

.361 

384 

.142 

.125 

-.050 

Avionics  -  K 

.089 

.297 

.039 

.047 

-.031 

Note,  n  =  202 
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variables.  Based  on  the  results  of  this  factor  analysis,  we  labeled  the  first 
factor  Knowledge  and  the  second  factor  Performance.  The  majority  of  the  tests 
that  load  .30  or  greater  on  Factor  1  are  knowledge  tests,  and  the  majority  of 
tests  that  load  .30  or  greater  on  Factor  2  are  Performance  tests.  Notice  that 
the  tests  that  load  above  .30  on  both  factors  load  highest  on  the  factor 
representing  the  testing  format,  either  knowledge  or  performance.  For  example, 
knowledge  Test  331  (AC  Theory)  loads  .54  on  Factor  1  and  .32  on  Factor  2. 

In  order  to  examine  whether  a  knowledge  test  versus  performance  test 
distinction  could  explain  aspects  of  school  performance,  we  created  both  a 
Knowledge  composite  variable  (by  taking  an  average  of  the  knowledge  tests)  and 
a  Performance  composite  variable  (by  taking  an  average  of  the  performance  tests). 
We  identified  three  phases  of  AE  training  by  examining  the  content  covered  by  the 
three  comprehensive  exams  and  created  composite  variables  based  on  the  three 
content  areas.  The  Phasel  composite  variable  represents  tests  301-341  which 
covered  mathematics  and  basic  electrical  theory.  The  Phase2  composite  variable 
represents  tests  352-391  which  covered  more  complex  topics  including  electronic 
circuitry,  generators  and  hydraulic  systems.  Finally,  the  Phase3  composite 
variable  represents  tests  402-451  which  covered  various  aviation  systems. 
Additional  composites  included  the  Suml  and  Sum2  variables.  For  Suml  we  took  a 
simple  average  of  all  the  knowledge  tests  that  loaded  .30  or  above  on  Factor  1. 
Similiarly  for  Sum2,  we  took  the  average  of  all  performance  tests  that  loaded  on 
Factor  2.  Since  the  AE  school  computed  the  FSG  using  a  63  as  the  minimum  score, 
we  calculated  our  own  final  grade  (FSG2)  using  initial  test  scores,  and  listed 
both  as  composite  variables.  We  used  the  factor  scores,  Factorl  and  Factor2,  as 
additional  composite  variables.  Table  10  lists  the  descriptive  statistics  for 
each  of  the  composite  variables  listed  above.  The  n  varied  for  the  composite 
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Tabic  lu 


Descnrtive  Statistics  for  AF  Composite  \'anablcs 


Composiic 

T\ 

Rel® 

.Mean 

Std  Dev 

Skew 

Kunosis 

Min 

Max 

Suml“ 

203 

.901 

82.133 

6.782 

-.043 

-.546 

65^8 

97.82 

Sum2^ 

202 

.804 

87.749 

6.518 

-.655 

-.248 

70.09 

99.54 

FSG 

22S 

.\/A 

83.178 

6.621 

-.663 

1.843 

53.90 

98.20 

FSG2^ 

202 

.900 

83.740 

5.736 

.074 

-  .452 

69.34 

98.20 

Know® 

203 

.895 

81.107 

6.981 

.(Ml 

-  i53 

63.96 

97.35 

Pcrf‘ 

202 

.656 

90.273 

4.368 

-.538 

.032 

76.55 

99.70 

Phase 

220 

.741 

87.263 

6.753 

-.430 

-.647 

68.66 

99,06 

Phasc2‘'' 

206 

.788 

80.384 

6.301 

-.475 

-.325 

61.88 

92.64 

Phases^ 

202 

.700 

86.770 

4.458 

-.129 

.033 

74.31 

98.21 

^These  arc  composiics  based  on  communatity  estimates  and  the  procedures  described  in  the  text. 
^5uml:  Average  of  all  knowledge  tests  with  Factor  1  loadings  greater  than  or  equal  to  .30 
^Sum2:  Average  of  all  performance  tests  v,iih  Factor  2  loadings  greater  than  or  equal  to  .30 
^FSG2;  Weighted  average  of  all  INTTLAL  test  scores  (See  Appendix  C  for  Test  Weight  Scheme.) 
^Kno^":  Average  of  all  knowledge  tests 
"Perf:  Average  of  all  performance  tests 
-  Phase  1:  Average  of  Tests  301  -  341 
“Phasc2:  Average  of  Tests  352  -  391 
‘Phase3:  Average  of  Tests  402  -  451 


variables  depending  on  the  number  of  tests  missing  for  each  case. 

Finally,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  11  presents  the  intercorrelations  between  all  composite 
variables.  Notice  that  FSG  and  FSG2  correlated  .99  showing  no  support  for  using 
initial  test  scores  to  compute  FSG. 

These  results  also  show  that  FSG  correlates  very  highly  with  the  knowledge 
composites:  Factor 1  (.90),  Suml  (.97),  and  Knowledge  (.97).  FSG  shows  lower 
correlations  with  the  performance  composites:  Factor2  (.54),  Sum2  (.78),  and 
Performance  (.77). 

The  strongest  correlation  with  Factor2  occurs  for  Sum2  (.91),  followed 
closely  by  Performance  (.86).  Sum2  and  Performance  also  correlate  very  highly 
(.90). 

Phasel  correlates  the  strongest  with  Suml  (.88)  suggesting  an  emphasis  on 
knowledge  tests  in  that  phase.  Phase2  correlates  most  strongly  with  Sum2  (.95), 
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Tabic  11 


Correlations  Among  AF  Composites  and  Factors 


Comp 

Factorl 

Factor2 

Suml* 

Sum2“ 

FSG 

FSG2' 

Know^ 

Perf* 

Phase 

Phasc2®  Phasc3^* 

Factorl 

Factor2 

.17 

Suml 

.95 

.45 

Sum2 

.51 

.91 

.72 

FSG 

.90 

.54 

.97 

.78 

- 

FSG2 

.91 

.54 

.98 

.79 

.99 

Know 

.95 

.42 

.98 

J7 

.97 

.97 

Perf 

.48 

.86 

.68 

.90 

.77 

.77 

.64 

... 

Phase  1 

.82 

.41 

.88 

.62 

.84 

.84 

.86 

.60 

Phasc2 

.60 

.81 

.77 

.95 

.83 

.83 

.77 

.84 

.66 

Phasc3 

.67 

.47 

.72 

.61 

.78 

.79 

.72 

.79 

.53 

.56 

®Suml:  Average  of  all  knowledge  tests  with  Factor  1  loadings  greater  than  or  equal  to  .30 

®Sum2:  Average  of  all  performance  tests  with  Factor  2  loadings  greater  than  or  equal  to  .30 

^FSG2:  Weighted  average  of  all  INTTIAL  test  scores 

^Know:  Average  of  all  knowledge  tests 

®Perf:  Average  of  all  performance  tests 

•Phasel:  Average  of  Tests  301  -  341 

-Phase2:  Average  of  Tests  352  •391 

^*Phasc3:  Average  of  Tests  402  -  451 


suggesting  an  emphasis  on  performance  in  that  phase.  Correlations  with  Phase3 
suggests  a  more  even  emphasis  of  knowledge  and  performance. 

Recommendations.  In  addition  to  FS6,  use  the  average  of  all  Sum2  computed 
as  an  average  of  the  following  performance  test  names  (numbers):  Maintenance 
(372),  AC/DC  Power  (382),  all  hydraulics  tests  (392  -  396),  Fuel  Systems  (402), 
Pitot  Static  (404),  Navigation  (411),  and  Jet  Ignition  (432). 
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Aviation  Ordnanceman  (AO) 

Description  of  Variables.  The  AO  school  revised  the  first  half  of 
the  curriculum  in  April  1991.  We  labeled  the  curriculum  before  this  date 
"Curriculum  A"  and  the  curriculum  after  this  date  "Curriculum  B".  Table 
12  lists  the  units  covered  by  both  Curriculum  A  and  B.  Unit  0  from 
curriculum  B  covers  units  0  and  1  from  Curriculum  A.  The  school  does  not 
test  students  on  unit  0  in  either  curriculum.  Units  3  and  4  from 
Curriculum  A  do  not  exist  as  separate  lessons  in  Curriculum  B.  However, 
units  2  and  1  from  Curriculum  B  include  the  material  covered  in  units  3 
and  4  from  Curriculum  A,  respectively.  Hence,  Curriculum  B  covers  the 
same  material  as  Curriculum  A.  However,  only  the  last  half  of  the 
curricula  share  the  same  lessons  and  tests. 


Tabic  12 

AO  School  Units  of  Instruction  from  Curricula  A  and  B 


Unit 

Topic 

0 

Curriculum  A  forior  to  Aoril  1991) 

Enlisted  Aviation  Basic  Training  and  Indoctrination 

1 

Introduction  and  Indoctrination 

2 

Electricity 

3 

Publications  and  Administration 

4 

Aircraft  Maintenance 

5 

Ammunition  and  Magazines 

6 

Ammunition  Ordnance 

7 

Aircraft  Guns 

8 

Guided  Missiles 

0 

Curriculum  B  fAoril  1991  to  Aoril  1992) 

Introduction  and  Indoctrination 

1 

Naval  Aviation  Maintenance 

2 

Electricity 

5 

Ammunition  and  Magazines 

6 

Ammunition  Ordnance 

7 

Aircraft  Guns 

8 

Guided  Missiles 
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The  school  administered  a  third  curriculum  to  classes  91210,  91240, 
and  91260  during  May  and  June  of  1991.  These  three  classes  piloted  a 
revised  second  half  of  the  curriculum  that  the  school  implemented  April 
24,  1992.  The  last  examinees  for  this  study  completed  the  AO  school  in 
March  1991.  Therefore,  only  classes  91210,  91240,  and  91260  completed  the 
third  curriculum.  We  did  not  analyze  this  data  from  the  third  curriculum 
because  we  collected  only  two  cases  from  these  three  classes,  and  because 
the  school  made  several  changes  between  the  pilot  curricula  they  completed 
and  the  curricula  the  school  actually  implemented  on  April  24,  1992. 

For  each  unit  of  instruction,  students  receive  practical  work  and  a 
written  knowledge  test.  Students  also  receive  a  within-course 
comprehensive  test  at  the  conclusion  of  Unit  2  and  a  final  comprehensive 
knowledge  test  at  the  conclusion  of  training.  In  addition.  Curriculum  B 
students  receive  a  within-course  comprehensive  performance  test.  The 
school  requires  a  minimum  passing  score  of  63%  for  all  tests.  Students 
who  do  not  pass  a  knowledge  progress  test,  the  within-course  comprehensive 
test  or  the  final  comprehensive  knowledge  test  receive  remedial 
instruction  and  a  retest.  The  school  assigns  the  minimum  passing  score  to 
those  passing  a  retest.  Students  who  fail  the  retest  must  appear  before 
the  Academic  Review  Board  (ARB).  Students  who  fail  practical  work  do  not 
receive  remediation  or  retesting,  but  they  do  continue  on  in  the 
curriculum  and  they  keep  their  failing  score.  There  is  no  retesting  for 
Curriculum  B  students  who  fail  the  within-course  comprehensive  performance 
test.  Students  who  fail  must  appear  before  the  ARB. 

We  entered  only  the  initial  test  scores  and  not  the  school-assigned 
score  (63%)  for  successful  retests.  The  student  action  code  variable 
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provides  the  graduation  status  of  the  student.  See  Appendix  B  for 
descriptions  of  the  various  disenrol Iment  categories. 

Table  13  lists  the  specific  tests  and  practical  work  used  in  the  AO 
school.  The  school  measures  all  tests  and  practical  work  on  a  scale  from 
0  to  100.  For  each  student,  the  school  computes  a  Final  School  Grade 
(FSG)  from  a  weighted  average  of  the  corresponding  Knowledge  tests  and 
Practical  work.  See  Appendix  D  for  practical  work  and  test  weights. 

Sample.  The  sample  consisted  of  290  students.  The  school 
administered  Curriculum  A  to  the  72  students  who  attended  the  AO  school 
before  April  1991.  Of  the  72  Curriculum  A  students,  24  had  an  incomplete 
data  set,  reducing  the  final  sample  available  for  analyses  to  48.  We 
conducted  no  analyses  on  these  data  because  the  sample  was  so  small  and 
because  this  was  the  older  curriculum.  Therefore,  the  final  sample 
consisted  of  213  cases  from  Curriculum  B  (5  cases  contained  missing  data). 

Results  and  Discussion.  We  computed  descriptive  statistics  for  each 
of  the  test  variables.  Table  14  lists  the  descriptive  statistics  for  all 
variables. 

Next,  we  conducted  a  factor  analysis  on  the  set  of  all  test 
variables.  The  scree  test  and  the  factor  loadings  indicated  two  factors. 
Based  on  the  results  of  the  factor  analysis,  we  labeled  the  first  factor 
Knowledge  and  the  second  factor  Practical.  All  tests  that  loaded  .30  or 
greater  on  Factor  1  were  Knowledge  tests  and  all  tests  that  loaded  .30  or 
greater  on  Factor  2  were  practical  work  with  the  exception  of  Maintenance 
-  K  and  Pubs/Maint  -  K.  Although  Maintenance  -  K  loaded  over  .30 


Tabic  13 


Descnriions  or  AO  Vanables 


Unit 


I 


1 

1 


I 


I 


1 

1 


1 

2 

2 


Test  No.  Desenpuon  (Name  -  T>pc  of  Test^) 


Curricuiuin  A  only 

111  Practical  Work:  Introduction  and  Indoctrination.  Introduction  to  the  AO  Field.  General  Ordnance  Safety. 

Tool  Control.  Introduction  to  Elcctricit>'.  Theory  of  a  Scries  Direct  Current  Circuit.  Conductors,  and 
Insulators.  Application  of  current  in  a  Direct  Current  Armament  Circuit.  Measuring  current.  Voltage,  and 
Resistance  with  a  Multimeter,  Introduction  to  Voltage  in  a  Series  Armament  Circuit,  Introduction  to 
Resistance  in  a  Scries  Armament  Circuit.  Application  of  Ohm’s  Law,  Relationship  and  Application  of 
current.  Voltage,  and  Resistance  in  a  Series  Armament  Circuit.  Measuring  current.  Voltage,  and  Resistance 
in  a  Scries  Direct  Current  Circuit  with  a  Multimeter.  (Intro  Elect  -  P) 

113  Knowledge  Test  See  Module  #111  for  description  of  test  material.  (Intro  Elect  -  K) 

121  Practical  Wort  Current.  Voltage,  and  Resistance  in  a  Parallel  Armament  Circuit  and  Measuring  it  with 
a  iMultimeter.  Troubleshooting  Parallel  Armament  Circuits.  Current.  Voltage,  and  Resistance  in  a 
Combination  Armament  Circuit.  Troubleshooting  and  Measuring  Combination  Armament  Circuits. 
Transformers.  Application  of  Transformers  in  an  Armament  Circuit.  Fitters  Relays.  Maintenance,  and 
Repair  of  Armament  Circuits.  Electrical  Control  Devices.  Protection  Devices  and  Symbols.  6E10  Training 
Device  Familiarization.  Aircraft  Troubleshooting  (6E11  Training  Device).  (Circuits  -  P) 

123  Koowkdge  Test  See  Module  #121  for  description  of  test  material.  (Circuits  -  K) 

201  PracticaJ  Wort  Wiring  C^e  and  Diagrams.  Aircraft  Armament  Circuits.  Suspension,  and  Releasing 

Equipment,  Introduction  to  the  A-4  Aircraft,  Aircraft  Maintenance  Manuals.  Aircraft  Armament  Test 
Equipment.  Arcraft  Preparation  and  Inspection.  Introduction  to  Arcraft  Armament  Troubleshooting 
Techniques,  Release  and  Control  Checks.  System  Troubleshooting  Techniques  and  Documentation.  (Intro 
Arcrafi  -  P) 

203  Knowiedge  Test  See  Module  #201  for  description  of  test  material.  (Intro  Arcraft  -  K) 

343  Knowledge  Test  Introduction  to  NAVSEA  Publications.  Administrative  Publications.  Ordnance  Safety  and 

Supply  Publications.  Introduction  to  NAVAR  Publications,  Loading  Manuals  and  CheckJisu.  Introduction 
to  Arcraft  Corrosion,  Corrosion  Control  Manual.  Theory  of  Corrosion.  (Pubs/Corrosion  -  K) 

Curriculum  B  only 

101  Practical  Wort  Naval  Aviation  Maintenance  Program  and  Discrepancy  Reporting  Program.  Planned 
Maintenance  Systems.  Maintenance  Data  Forms,  Foreign  Object  Damage,  Tool  Control,  Common  Hand 
Tools,  Arcraft  Hardware,  Torquing  and  Safetying,  Shop  Project.  (Maintenance  -  P) 

110  Knowledge  Test  See  Module  #101  for  description  of  test  material.  (Maintenance  -  K) 

102  Practical  Work:  Corrosion  Control  Manuals,  Corrosion  Theory,  Corrosion  Control  Program,  Corrosion 
Corrective  Maintenance,  Corrosion  Inspections,  Paint  and  Touch-up,  Preservation/ 

Depreservation.  Emergency  Corrosion  Treatment,  Aircraft  Qeaning.  (Corrosion  -  P) 

120  Knowledge  Test  See  Module  #102  for  description  of  test  material.  (Corrosion  -  K) 

221  Practical  Work:  Electrical  Safety,  Electrical  Fundamentals,  Ohm's  Law,  Measuring  Dectrical  Values, 
Protective  Control  Devices.  (Electrical  -  P) 

210  Knowiedge  Test  See  Module  #221  for  description  of  test  material.  (Electrical  •  K) 


“Refers  to  Practical  work  (P)  or  Knowledge  lest  (K) 
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Table  13  (Coniinucd) 


L  nil  \  ar.  No.  Descnption  iNamc  -  Tvpc  oi  Tcsi^) 


1-2 


1-2 


5 

6 


6 

6 


6 

7 


7 

8 


222  Pncticml  Wort  Introduction  to  Publications.  Wire  Maintenance.  Troubleshooting.  Aircraft  Armament  Test 

Et^uipment.  Loading  Manual  (Chapters  1-^)  and  NA-Ol-700.  Aircraft  Armament  Sv'stems.  introduction  to 
the  Weapons  Systems  Trainer,  Weapons  System  Functional  Checks  and  Troubleshooting.  (Pubs/Maint  - 
P) 

220  KocNvledge  Test  Sec  Module  #222  for  description  of  test  maienal.  (Pubs/Maint  •  K) 

230  Within-oourse  Comprehensive  Knowledge  Test  covers  test  material  in  Modules  #101  through  #220 
(Comp  1-  K) 

231  Within-ooursc  Comprehensive  Performance  Test  covers  test  material  in  Modules  #101  through  #220 
(CTomp  2  -  P) 

Curricula  A  and  B 

501  PracticaJ  Work  Ammunition  Stowage  Ashore,  at  adv-anced  Bases,  and  Afloat.  Identification  and  Types  of 
.Magazines  Ashore.  .Magazine  Temperatures  and  Safety*  Devices.  Ammunition  Handling  and  Transponation 
Equipment.  Procedures  for  Ordering  and  Turning  in  Ammunition.  (Ammunition  -  P) 

503  Knowledge  Test  See  Module  #501  for  description  of  test  material.  (Ammunition  -  K) 

Practical  Wort  Introduction  to  Airborne  Ordnance.  General  Purpose  Bombs,  Introduction  to  Freefall  and 
Retarded  Airborne  Weapons.  Introduction  to  Aircraft  Mechanical  Bomb  Fuses  and  Boosters.  Introduction 
to  Electrical  Bomb  Fuses.  Practice  Bombs  and  Signals.  Laser  Guided  Bomb  AGM-123  Skipper  Missile, 
Cluster  Bomb  Lnit  (CBU).  Firebombs  and  Associated  Components,  Special  Purpose  Munitions,  Aiming 
Wire.  (Bombs  •  P) 

613  KiKwicdge  Test  Sec  Module  #611  for  description  of  test  material.  (Bombs  -  K) 

621  Practical  Work  Introduction  to  Airborne  Rockets.  Rocket  Components.  Introduction  to  Pyrotechnics, 

Introduction  to  Ammunition  Suspension  Equipment.  Aircraft  Arming  Signals.  Conventional  Weapons 
Loading/Unloading,  Inicgratcd  Rapid  Rearming  System  (IRRS).  (Rockets  -  P) 

623  Koowiedge  Test  Sec  Module  #621  for  description  of  test  material.  (Rockets  -  K) 

701  Practical  Work  Introduction  to  20MM  Aircraft  Guns.  Aircraft  Ammunition  20MM.  Introduction  to  the 

M61A1  Vulcan  Gun.  Nomenclature  and  Function  of  the  Major  Components  of  the  M61A1  Vulcan  Gun. 
M61A1  Vulcan  Gun  linkless  Ammunition  Loading  System,  Introduction  to  the  A-7E,  F-14,  F/A-18  Gun 
System,  General  Aircraft  Safety.  LALS  Loading/Downloading.  (Aircraft  Guns  -  P) 

703  Knowledge  Test  See  Module  #701  for  description  of  test  material.  (Aircraft  Guns  -  K) 

Practical  Work  Introduction  and  Function  of  Air-Launched  Guided  Missiles.  Introduction  and  Function 
of  the  Shrike  Missile  and  its  Components,  Introduction  and  Function  of  the  Sidewinder  Missile  and  its 
Components.  Introduction  and  Function  of  the  Sparrow  III  Missile  and  its  Components.  Intro  and  Function 
of  the  Standard  Arm  Missile.  Introduction  and  Function  of  the  Phoenix  Missile,  Introduction  and  Function 
of  the  Walleye  Weapon.  Introduction  to  the  New  Missiles,  Missile  Handling  Equipment,  Missile  Launching 
Equipment,  Procedures  and  Missile  Safety,  Weapons  Assembly,  Inspection,  Loading,  Arming,  Dearming, 
Downloading,  and  Disassembly.  (Missiles  -  P) 


®  803  Knowledge  Test  See  Module  #801  for  description  of  test  material.  (Missiles  -  K) 

900  Final  Comprehensive  Test 

(Curriculum  A)  -  Entire  course  comprehensive  test. 

(Curriculum  B)  -  Covers  from  Module  #231  through  #803.  (Final  Comp.) 

^Refers  to  Practical  work  (P)  or  Knowledge  test  (K) 
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Table  14 


Descnmive  Statistics  for  AO  Variables  -  Cumculum  R 


Variable 

Mean 

Sid  Dev 

Skew 

Kunosis 

Min 

Max 

Maintenance  -  P 

218 

92.241 

9.042 

-4.874 

9.042 

8.10 

100.00 

Maintenance  -  K 

218 

77341 

11.000 

•  .444 

-  .165 

46.00 

98.00 

Corrosion  -  P 

218 

95.033 

7.847 

-5348 

39.907 

26.90 

100.00 

Corrosion  -  K 

217 

85.862 

9337 

-.382 

-.394 

56.00 

100.00 

Electrical  -  P 

217 

92.179 

6.952 

-2.362 

10319 

4430 

100.00 

Electrical  -  K 

216 

84.704 

8.062 

-.607 

.134 

60.00 

100.00 

Pubs/Maini  -  P 

216 

92.991 

7.675 

-3.474 

19.200 

3530 

100.00 

Pubs/Maint  -  K 

218 

82.787 

10.689 

-  371 

-.305 

54.00 

100.00 

Comp.  1  •  K 

216 

83.697 

7.854 

-  .374 

-  .282 

61.30 

100.00 

Comp.  1  -  P 

218 

95.185 

8.400 

-7.088 

74.034 

1.00 

100.00 

Ammunition  -  P 

216 

93.076 

9.790 

-3.882 

26.177 

9.00 

100.00 

Ammunition  -  K 

216 

82300 

9.904 

-  .693 

.212 

50.00 

100.00 

Bombs  -  P 

214 

94.031 

7.182 

-2.774 

11322 

51.40 

100.00 

Bombs  -  K 

214 

84.738 

10385 

-.890 

.429 

48.00 

100.00 

Rockets  -  P 

214 

93.009 

10.489 

-4.688 

32.144 

00,70 

100.00 

Rockets  -  K 

214 

85.350 

10.904 

-1.015 

1.429 

40.00 

100.00 

Aircraft  Guns  -  P 

214 

95.476 

10.008 

-4.968 

32.460 

9.10 

100.00 

Aircraft  Guns  -  K 

214 

81.196 

12.681 

-.774 

1.871 

20.00 

100.00 

Missiles  -  P 

214 

94382 

7.872 

-2.616 

9.694 

45.60 

100.00 

Missiles  >  K 

214 

82.766 

11.723 

-.421 

-.489 

48.00 

100.00 

Final  Comp. 

214 

86.785 

10.181 

-.431 

-.992 

64,00 

100.00 

on  both  Factors, 

it 

loaded 

considerably 

higher  on 

Factor 

1. 

Table  15 

lists  the  communal i ties,  factor  loadings,  and  factor  score  coefficients 
for  each  of  the  variables. 

In  order  to  examine  whether  a  knowledge  test  versus  performance  test 
distinction  could  explain  aspects  of  school  performance,  we  created  two 
composite  variables:  one  based  on  the  sum  of  the  knowledge  tests  and  the 
other  based  on  the  sum  of  the  practical  work.  We  identified  two  phases  of 
the  AO  training  based  on  the  material  and  tests  covered  by  the  two 
comprehensive  exams.  The  Phasel  composite  variable  consisted  of  an 
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Table  15 


rommunniities.  Factor  l  oadings  and  Factor  Score  Coefficients  for  AO  V'ariables  «  Curriculum  R 


Factor  Loadings  Factor  Score  Coefficients 


Variable 

Communality 

Factor  1: 
Knowledge 

Factor  2: 
Practical 

Factor  1: 
Knowledge 

Factor  2: 
Practical 

.Maintenance  -  P 

.191 

.260 

.351 

.020 

.074 

Maintenance  -  K 

.431 

J571 

.323 

.127 

.035 

Corrosion  -  P 

.227 

.105 

.465 

.006 

.051 

Corrosion  -  K 

.354 

.544 

.240 

.112 

.025 

Electrical  -  P 

.121 

.197 

.289 

.006 

.079 

Electrical  -  K 

.289 

.514 

.158 

.074 

-.001 

Pubs /Maim  -  K 

.247 

.214 

.449 

.008 

.090 

Pubs/Maint  -  P 

.jA6 

i34 

.247 

.105 

.008 

Compl  -  K 

i70 

.710 

.257 

.280 

-.086 

Compl  -  P 

.045 

.117 

.179 

.011 

.008 

.Ammunition  -  P 

.148 

.143 

.357 

-.011 

.067 

.Ammunition  -  K 

.296 

.531 

.119 

.111 

.008 

Bombs  -  P 

.767 

.032 

.875 

-.291 

.727 

Bombs  -  K 

.426 

.641 

.124 

.180 

-.040 

Rockets  -  P 

.304 

.215 

.508 

.032 

.040 

Rockets  -  K 

.353 

.582 

.122 

.128 

-.024 

Aircraft  Guns  •  P 

.101 

.054 

.314 

-.017 

.072 

Aircraft  Guns  -  K 

.387 

.578 

.230 

.127 

-.008 

Missiles  -  P 

.221 

.291 

.369 

.036 

.042 

.Missiles  -  K 

.294 

.496 

.218 

.081 

.046 

Final  Comp. 

.187 

.417 

.116 

.071 

-.024 

Note,  n  =  213 


average  of  all  knowledge  tests  and  practical  work  up  to  and  including  the 
first  comprehensive  exam.  The  Phase2  variable  consisted  of  an  average  of 
all  knowledge  work  and  practical  work  up  to  and  including  the  second 
comprehensive  exam.  Additional  composites  included  the  Suml  and  Sum2 
variables  which  took  a  simple  average  of  all  the  tests  that  loaded  .30  or 
above  on  Factor  1  and  Factor  2,  respectively.  Since  the  AO  school 
computed  the  FSG  using  a  63  as  the  minimum  score,  we  calculated  our  own 
final  grade  (FSG2)  using  the  initial  test  scores,  and  listed  both  as 
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composite  variables.  We  used  the  factor  scores,  Factorl  and  Factor2,  as 
additional  composite  variables.  Table  16  lists  the  descriptive  statistics 
for  each  of  the  composite  variables  listed  above.  The  n  varied  for  the 
composite  variables  depending  on  the  number  of  students  with  an  incomplete 
data  set. 

Next,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  17  presents  the  composite  variable  intercorrelations. 
Since  FSG  and  FSG2  correlate  .97,  this  offers  no  support  for  using  initial 
test  scores  to  compute  FSG. 

Knowledge  correlates  highest  (.95)  with  Factorl,  followed  closely  by 
Suml  (.94)  and  FSG  (.89).  Similarly,  Practical  correlates  highest  (.86) 
with  Factor2,  followed  closely  by  Sum2  (.82).  These  findings  support 
using  the  logical  combinations  of  tests  to  assess  the  dimensions  reflected 
in  the  factor  analyses. 


Table  16 

.Descriptive  Statistics  for  AO  Composite  Variables  -  Curriculum  B 


Composite 

,\ 

Rel* 

.Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Suml° 

213 

.856 

83.106 

6.718 

.010 

-.428 

6538 

98.20 

Sum2® 

214 

.755 

93.980 

4,499 

-1-585 

3.015 

75.97 

99.84 

FSG 

218 

N/A 

86.128 

5.422 

-.036 

-.605 

72.90 

98.20 

FSG2'^ 

213 

.846 

85.445 

5.720 

-.045 

-.419 

69.79 

9830 

Knowledge* 

213 

.856 

83.442 

6.532 

-.017 

-396 

6535 

9836 

Practical^ 

214 

.724 

93.918 

4.082 

-1310 

1.792 

79.18 

99.62 

Phase  1^ 

215 

.778 

88.210 

4.847 

-.486 

.153 

73.64 

9836 

Phase2^ 

214 

.795 

88J59 

5.492 

-.496 

.106 

71.10 

99.64 

®Suml:  Average  of  tests  110,  120,  210,  220,  230,  503,  613,  623,  703,  803 

®Suin2:  Average  of  tests  101, 102,  222,  501,  611,  621,  701,  801,  900 

°FSG2:  Final  School  Grade  calculated  using  initial  test  scores 

•Know:  Average  of  all  knowledge  tests 

^Prac:  Average  of  all  praaical  work 

*Phascl:  Average  of  tests  lOl  -  231 

^Phasc2:  Average  of  tests  501  -  900 


31 


Table  17 


Correlations  Among  AO  Composites  and  Factors  -  Curriculum  B 


Comp 

Factorl 

Factor2 

Suml® 

Sum2^ 

FSG 

FSG2' 

Know^ 

Prac* 

Phasel^  Phasc2* 

Factorl 

Factor2 

.08 

Suml 

.94 

.34 

- 

$um2 

.40 

.82 

.55 

- 

FSG 

.89 

.42 

.95 

.66 

FSG2 

.93 

.39 

.97 

.62 

.97 

— 

Know 

.95 

.99 

.58 

.96 

.98 

Prac 

.33 

.86 

.51 

.93 

.61 

i8 

.51 

- 

Phasel 

.76 

.54 

.85 

.63 

.80 

.86 

.83 

.70 

Phase2 

.76 

.55 

.85 

.82 

.92 

.88 

.87 

.73 

.67 

*Suml:  Average  of  tests  110.  120.  210,  220.  230,  503.  613,  623.  703  and  803 

‘>Sum2:  Average  of  tests  101.  102.  222,  501.  611,  621,  701,  801.  and  900 

~FSG2:  Final  School  Grade  calculated  using  initial  test  scores 

^Knou"  Average  of  ail  knowledge  tests 

®Prac:  Average  of  all  practical  work 

^Phasel:  A.verage  of  tests  101  -  231 

® Phase 2:  Average  of  tests  501  -  900 


In  reviewing  the  results  with  phase  variables,  we  find  that  both 
Phasel  and  Phase2  correlate  more  highly  with  knowledge  composites  than 
with  performance  composites.  However,  those  more  moderate  correlations 
(.54  to  .87)  are  not  as  supportive  of  a  unidimensional  relationship  as  are 
the  results  for  the  Knowledge  and  Practical  Composites. 

Finally,  FS6  correlates  .61  with  Practical  and  .96  with  Knowledge. 
These  findings  support  using  FSG  as  an  indicator  of  performance  on 
knowledge  tests  but  suggest  using  a  separate  indicator  for  performance  on 
practical  work. 

Recommendation.  In  addition  to  FSG,  use  an  average  of  all  Practical 

tests. 
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Aviation  Structural  Mechanic  (Structures)  (AMS) 

Description  of  Variables.  The  AMS  school  divides  the  curriculum  and 
testing  plan  into  6  units.  The  school  evaluates  students  on  Knowledge, 
Performance,  and  Practical  Work  Test  Scores,  all  of  which  can  range  from  0-100. 
The  school  sets  63  as  the  minimum  passing  score  for  each  test.  Students  who  do 
not  achieve  the  minimum  passing  score  take  one  retest  per  test.  For  students 
passing  the  retest,  the  AMS  school  assigns  a  score  of  63.  For  students  who  fail 
the  retest,  an  Academic  Review  Board  assigns  night  study  and  allows  the  student 
to  continue  taking  retests  until  they  achieve  the  minimum  passing  score.  RGI 
entered  only  the  initial  test  scores  into  the  database.  We  also  entered  the 
number  of  retests  on  each  test.  Student  Action  Code  reflects  graduation  status 
or  disenrollment  category  (See  Appendix  B).  Table  18  describes  the  units  and 
corresponding  tests. 

For  each  student,  the  school  computes  unit  grades  as  weighted  averages  of 
the  appropriate  Knowledge,  Performance,  Practical  Work,  and  Comprehensive  test 
scores.  The  school  then  computes  a  Final  School  Grade  (FSG)  as  a  weighted 
average  of  the  unit  grades.  The  school  computes  the  Unit  Grades  as  shown  in 

Table  19. 

Sample.  The  sample  consisted  of  a  total  of  307  students. 

Results  and  Discussion.  Table  20  displays  descriptive  statistics  for  the 
AMS  variables.  Factor  analysis  indicated  two  clear  factors,  one  defined  by  the 
knowledge  tests  ("Knowledge"),  the  other  defined  by  the  performance  tests  and 
practical  work  ("Performance").  The  performance  test  covering  Maintenance  and 
Material  Documentation  showed  the  only  exception  to  this  pattern.  Table  21 
displays  the  communal ities,  factor  loadings,  and  factor  score  coefficients  for 
the  AMS  variables,  grouped  by  knowledge  and  performance  test/practical  work. 
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Table  IS 

AMS  School  Instruciional  Units  and  Vanables 


L  nit  \  anaoie  Dcscnpiion  iNamc  -  i>-pc  oi  Fcs;'^) 


1  101 
103 


Knowledge  lesi  covenng  a  Basic  iniroducuon  to  the  AMS  school  (Basic  Intro  -  K) 
Practical  work  covering  the  Introduction  (Basic  Intro  -  PW) 


201  Knowledge  test  covenng  Maintenance  and  Material  Documentation  (Documentation  •  K") 

202  Performance  test  covenng  Maintenance  and  Material  Documentation  (Documentation  -  P) 


3 


Sectioo  1 

311  Kncn^iedgc  test  covenng  Corrosion  Control  (Corrosion  Control  -  K) 

313  Practical  work  covenng  Corrosion  Control  (Corrosion  Control  -  PW) 

Sectioo  2 

321  Knowledge  test  covenng  Aircraft  Painting  and  all  related  areas  (Aircraft  Painting  -  K) 

322  Performance  test  covenng  Corrosion  Control  and  Painting  (Corrosion  Control  &  Painting  -  P) 

323  Practical  work  covenng  Aircraft  Painting  (Aircraft  Painting  -  PW) 


4 


5 


6 


Scctioa  1 

411  Knowledge  test  covenng  Basic  Aircraft  Construction,  Aircraft  Structural  Materials,  the  Fundamentals  of  Drawing 
Interpretation,  the  Bend  allowance  for  90  degree  Bends,  the  Development  of  a  Flat  Layout.  Flat  Layout  on  .Metal. 
Cutting  Aircraft  Structural  Metal,  Drilling  Structural  Metals,  and  how  to  Cut  Structural  Metals  (Basic  Aircraft 
Construction  -  K) 

413  Practical  work  covenng  topics  in  knowledge  test  #411  (Basic  Aircraft  Construction  -  PW) 

Sectioo  2 

421  Knowledge  test  covenng  Forming  Aircraft  Structural  Metals,  the  Installation  of  Permanent  Fasteners,  the  Layout  of 
Riveted  Joints.  Removal  of  Rivets  and  Scrap  Metal.  Layout  and  Drill  Rivet  Joints,  and  Countersinking  (Forming 
Aircraft  •  K") 

422  Performance  test  covering  topics  in  knowledge  tests  #411  and  #421  (Basic  Aircraft  Construction  &  Forming  -  P) 

423  Practical  work  covering  topics  in  knowledge  test  #421  (Forming  Aircraft  -  PW) 

Scctioa  3 

431  Knowledge  test  covering  Repair  of  Externa!  Skin.  Repair  External  Skin  by  Patching.  Installation  of  Semipermanent 
Fasteners,  the  Installation  of  Tumlock  Fasteners,  and  Making  an  Access  Panel  (Repair  External  Skin  -K) 

433  Practical  work  covenng  topics  in  knowledge  test  #431  (Repair  External  Skin  -  PW) 

Sectioo  4 

441  Knowledge  test  covering  the  Repair  of  Internal  Structures,  the  Repair  of  a  Damaged  Rib.  the  Repair  of  a  Damaged 
•U*  Channel,  and  Nondestructive  Metal  Inspections  (Repair  Internal  Structures  -  K) 

442  Performance  test  covering  topics  in  knowledge  tests  #431  and  #441  (Repair  External  and  Internal  -  P) 

443  Practical  work  covenng  topics  in  knowledge  test  #441  (Repair  Internal  Structures  -  PW) 

Section  1 

511  Knowledge  test  covering  the  Fabrication  of  Reinforced  Plastics,  the  Repair  of  Reinforced  Plastic,  Transparent  Plastic. 

and  the  Repair  of  Sandwich  Construction  (Reinforced  Plastics  •  K) 

513  Practical  work  covering  Aircraft  Non-Metallics  (Aircraft  Non-Metallics  -  PW) 

Sectioo  2 

521  Knowledge  test  covering  the  Fabrication  of  a  Circular  Patch,  an  Introduction  to  Composite  Materials,  and  a  Composite 
Material  Repair  Lab  (Circular  Patch  -  K) 

Sectioo  1 

611  Knowledge  test  covering  Aircraft  Flight  Controls  and  Mechanisms.  Planned  Maintenance  System.  Aircraft  Servicing. 

Fuel  Cell  Repair,  Lubrication  of  an  Aircraft,  and  the  Service  of  a  Pneumatic  System  (Flight  Controls  -  K) 

613  Practical  work  covering  topics  in  knowledge  test  #611  (Flight  Controls  -  PW) 

Sectioii  2 

621  Knowledge  test  covering  Aircraft  Jacking,  the  Maintenance  of  Wheels  and  Hres.  Raise  and  Lower  Complete  Aircraft. 

Remove  and  Replace  Wheel  Assembly,  the  Maintenance  of  Arresting  Hooks.  Remove  and  Replace  a  Hook  Point,  and 
Flight  Deck/Linc  Safety  (Aircraft  Jacking  -  K) 

623  Practical  work  covering  topics  in  knowledge  test  #621  (Aircraft  Jacking  -  PW) 

7(X)  Final  Comprehensive  test  given  at  the  end  of  Unit  6  (Comprehensive  -  C) 


^Kcy  to  Type  of  Test:  K  -  knowledge,  P  -  performance,  PW  -  practical  work.  C  -  comprehensive 
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Tabic  iv 

A.MS  School  Unit  Grades  and  Test  Weights 


Wci&hicd  Test  Componcnii 


Unit  1: 

Unit  2: 

Unit  3  Sec.  1: 
Unit  3  Sec.  2: 
Unit  4  See.  1: 
Unit  4  See.  2: 
Unit  4  See.  3: 
Unit  4  See.  4: 
Unit  5: 

Unit  6  See.  1: 
Unit  6  Sec.  2: 


.60  X  (#101) 
.40  X  (#201) 
.70  X  (#311) 
.25  X  (#321) 
.60  X  (#411) 
.30  X  (#421) 
,40  X  (#431) 
.20  X  (#441) 
.30  X  (#511) 
.60  X  (#611) 
.60  X  (#621) 


T  .40  X  (#103) 
+  .60  X  (#202) 
+  .30  X  (#313) 
+  .50  X  (#322) 
+  .40  X  (#413) 
+  -50  X  (#422) 
+  .60  X  (#433) 
+  .50  X  (#442) 
+  .30  X  (#513) 
+  .40  X  (#613) 
+  .40  X  (#623). 


+  .25  X  (#323) 

+  .20  X  (#423) 

+  .30  X  (#443) 
+  .40  X  (#521) 


Note.  The  school  computes  FSG  from  the  Unit  Grades  as:  ,  o  -.n 

FSG  *  03  X  (Unit  1)  +  .07  x  (Unit  2)  +  .07  x  (Unit  3  See.  1)  +  .10  x  (Unit  3  Sec.  2)  +  .15  x  (Unit  4  See.  1)  +  .15  x  (Unit  4  Sec.  2)  + 
.075  X  (Unit  4  Sec.  3)  +  .075  x  (Unit  4  Sec.  4)  +  .07  x  (Unit  5)  +  .07  x  (Unit  6  Sec.  1)  +  .07  x  (Unit  6  Sec.  2)  +  .07  x  (#700). 


Table  20 

Desenmive  Statistics  for  AMS  Variables 


N’ariable 

n 

Mean 

Std  Dev 

Skew 

Kurtosis 

Minimum  Maximum 

Basic  Iniro  -  K 

307 

a5,114 

9.263 

.1385 

6301 

2730 

100.00 

Basic  Intro  -  PW 

307 

94.199 

4.460 

-1.786 

5.247 

70.00 

100.00 

Documentation  -  K 

307 

77.323 

11.615 

-.806 

.722 

29.10 

Documentation  -  P 

307 

88.651 

11351 

-1.037 

.202 

48.00 

100.00 

(Corrosion  Control  -  K 

305 

85.929 

7.819 

-.619 

.071 

59.10 

Corrosion  Control  •  PW 

305 

92.898 

4.606 

-1.097 

2.069 

72.00 

100.00 

Aircraft  Painting  -  K 

305 

83.975 

8.782 

-.676 

.161 

55.70 

100.00 

Corrosion  Control  &  Painting  -  P 

305 

89,003 

6.974 

-1.257 

2.984 

53.00 

100.00 

Aircraft  Painting  -  PW 

305 

88.480 

7.308 

-1.485 

4.021 

48.00 

100.00 

Basic  Aircraft  Construction  -  K 

305 

81-545 

9.661 

-300 

.008 

49.20 

100.00 

Basic  Aircraft  Construction  -  PW 

305 

89.306 

7.026 

-1.957 

7.301 

42.10 

Forming  Aircraft  -  K 

305 

77.193 

11.071 

-  .398 

-  .167 

38.80 

Basic  Aircraft  Construction  &  Forming  -  P 

303 

83378 

10.205 

-1.750 

10.049 

6.00 

Forming  Aircraft  -  PW 

303 

73.833 

8.707 

.251 

-  1.148 

6030 

100.00 

Repair  External  Skin  -  K 

303 

82.483 

9.110 

-.462 

-  .181 

56.40 

100.00 

Repair  External  Skin  -  PW 

302 

76.092 

6.610 

-.099 

-  .490 

56.00 

Repair  Internal  Structures  -  K 

302 

86.072 

12.478 

-1.322 

2.934 

24.20 

100.00 

Repair  External  and  Internal  -  P 

302 

79.801 

9.719 

-.428 

-  .199 

46.00 

Repair  Internal  Stniaures  -  PW 

302 

80.058 

6391 

-.195 

-.266 

63.00 

96.00 

Reinforced  Plastics  -  K 

301 

84.066 

8332 

-.643 

.623 

51.20 

100.00 

Aircraft  Non-Metallics  -  PW 

301 

85.642 

7.459 

-.489 

.322 

63.00 

Circular  Patch  -  K 

301 

82346 

9.058 

-.489 

.060 

53.10 

100.00 

Flight  Controls  -  K 

301 

80.452 

9.497 

-.302 

-  .292 

53.20 

Flight  Controls  -  PW 

300 

83317 

7.316 

.188 

.365 

63.00 

100.00 

Aircraft  Jacking  -  K 

300 

90.484 

7.300 

-2.791 

16.879 

36.90 

Aircraft  Jacking  -  PW 

300 

84.838 

7.684 

-.102 

.299 

63.00 

100.00 

Comprehensive  -  C 

300 

75.675 

7.667 

-.159 

.474 

44.60 

100.00 
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Table  21 

rommunaiities,  Factor  Loadings,  and  Factor  Score  Coefficients  for  AMS  Variables 


Factor  i.o3dings _  Factor  ^core  Coefficients 


N'ariablc 

Communaliiy 

Factor  1; 
Knowledge 

Factor  2: 
Pcrf/Prac 

Factor  1: 
Knowledge 

Factor  2: 
Pcrf/Prac 

Basic  Iniro  -  K 

.372 

.596 

.089 

.107 

-.008 

Documentation  -  K 

.308 

-549 

.077 

.101 

-.041 

Corrosion  Control  -  K 

.359 

.593 

.085 

.113 

-.040 

Aircraft  Painting  -  K 

.458 

.634 

.235 

.123 

.042 

Basic  Aircraft  Construction  -  K 

.383 

.592 

.180 

.101 

.010 

Fonnmg  Aircraft  -  K 

.435 

.629 

.199 

.121 

.017 

Repair  External  Skin  -  K 

.236 

.478 

.087 

.070 

-.008 

Repair  internal  Structures  •  K 

.218 

.458 

.089 

.064 

-.013 

Reinforced  Plastics  •  K 

.424 

.636 

.141 

.127 

-.013 

Circular  Patch  -  K 

.261 

.502 

.093 

.085 

-.025 

Flight  Controls  -  K 

.460 

.653 

.183 

.143 

.002 

Aircraft  Jacking  -  K 

.214 

.429 

.173 

.041 

.028 

Comprenensivc  -  C 

.580 

.756 

.089 

.224 

-.096 

Basic  Intro  -  P\V 

.130 

-.042 

.358 

-.036 

.136 

Documentation  -  P 

.154 

.392 

.031 

.051 

-.023 

Corrosion  Control  -  PW 

.055 

.210 

.105 

.026 

.021 

Corrosion  Control  &  Painting  -  P 

.155 

.207 

.334 

-.001 

.100 

.-\ircran  Painting  •  PW 

.081 

.013 

.284 

-.012 

.085 

Basic  Aircraft  Construction  -  PW 

.093 

.087 

.293 

-.008 

.084 

Basic  Aircraft  Const.  &  Forming  -  P 

.157 

.182 

.352 

-.015 

.109 

Forming  Aircraft  -  PW 

.329 

.175 

S46 

-.039 

.228 

Repair  External  Skin  -  PW 

.296 

.175 

SIS 

-.026 

.206 

Repair  External  and  Internal  -  P 

.250 

.021 

SOO 

-.047 

.195 

Repair  Internal  Structures  -  PW 

.171 

.117 

.397 

-.025 

.128 

Aircraft  Non-Meta  Hies  -  PW 

.270 

.162 

.494 

-.035 

.202 

Flight  Controls  -  PW 

.088 

.102 

.278 

-.005 

.075 

.Aircraft  Jacking  -  PW 

.095 

.066 

.302 

-.016 

.092 

Note.  N  =  300. 


To  better  represent  the  factors  using  empirical  composites,  we  developed 
the  following  measures.  Suml  refers  to  the  mean  of  the  variables  with  Factor  1 
loadings  greater  than  or  equal  to  .30.  Sum2  refers  to  the  mean  of  the  variables 
with  Factor  2  loadings  greater  than  or  equal  to  .30.  FSG  indicated  the  school’s 
Final  School  Grade,  while  FSG2  refers  to  the  Final  School  Grade  calculated  using 
the  initial  test  scores.  Finally,  we  defined  Knowledge  as  the  mean  of  all 
knowledge  tests,  and  Performance  as  the  mean  of  all  performance  tests  and 
practical  work. 
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Table  22  shows  descriptive  statistics  on  these  composite  variables.  Next, 
we  computed  Pearson  correlation  coefficients  for  the  composite  variables.  Table 
23  shows  correlations  among  the  composites  and  the  factor  scores.  Since  FSG  and 
FSG2  correlate  .99,  this  offers  no  support  for  using  initial  test  scores  to 
compute  FSG,  Knowledge  and  Suml  correlate  .98  with  Factorl,  followed  by  FSG 
(.84).  Similarly,  Performance  correlates  .97  with  Factor  2  followed  by  Sum2 
(.96),  Finally,  FSG  correlates  .69  with  Performance  and  .90  with  Knowledge. 
These  findings  support  using  FSG  as  an  indicator  of  performance  on  knowledge 
tests  but  suggest  using  a  separate  indicator  for  performance  on  practical  work. 

Recommendations.  In  addition  to  Final  School  Grade,  use  Performance  (the 
average  of  the  performance  tests  and  practical  work). 


Table  22 

Descriptive  Statistics  for  AMS  Composite  VariaM^ 


Composite 

Rel® 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Saml“ 

,88 

83.038 

5.825 

.099 

-355 

68.28 

96.64 

Sum2= 

.68 

83.064 

4.020 

..093 

-.347 

70.83 

9230 

FSG 

N/A 

83383 

4.300 

-.090 

.637 

64.40 

94.60 

FSG2'^ 

.86 

83373 

4.197 

.186 

-.299 

72.04 

94.27 

Know® 

.88 

83.038 

5.825 

.099 

-355 

6838 

.  96.64 

Perf^ 

.70 

84.092 

3.607 

-.166 

-.211 

73.71 

92.88 

Note.  N  *  300. 

*Thcsc  arc  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^Suml:  Average  of  Tests  with  Factor  1  loadings  greater  than  or  equal  to  .30 
®Sum2:  Average  of  Tests  with  Factor  2  loadings  greater  than  or  equal  to  JO 
^FSG2:  Weighted  average  of  all  initial  test  scores  (Sec  Table  19  for  test  weights) 

•Know:  (Sum  of  Tests  101.  201,  311.  321,  411.  421, 431,  441, 511, 521,  611,  621.  700)/13 
^Perf:  (Sum  of  Tests  103,  322,  413,  323, 422,  423,  433,  442,  443,  513,  613,  623,  202,  313)/14 
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Tabic  23 

Correia t ions  Among  AMS  Composiies  ano  Factors 


Composite  “ 

Faciorl 

F3Ctor2 

Suml 

Sum2 

FSG 

FSG2 

Know 

Factor  2 

.11 

Suml*^ 

.98 

.25 

Sum2‘= 

.25 

.96 

.38 

FSG 

.84 

.58 

.90 

.69 

FSG2° 

.85 

.90 

.68 

.99 

Know® 

.98 

.25 

1.00 

.38 

.90 

.91 

Perf' 

.24 

.97 

.37 

.97 

.69 

.69 

.37 

Note.  N  =  300. 

^These  are  composites  based  on  communaliry  estimates  and  the  procedures  described  in  the  text. 
“Suml:  Average  of  Tests  wh  Factor  1  loadings  greater  than  or  equal  to  .30 
"Sum2:  Average  of  Tests  ^vith  Factor  2  loadings  greater  than  or  equal  to  .30 
-rSG2:  Weighted  average  of  all  initial  test  scores  (Sec  Table  19  for  test  weights) 

^Kno^"  (Sum  of  Tests  lOT  201.  311.  321.  411.  421.  431.  441.  511.  521.  611.  621.  700)713 
-Perf:  (Sum  of  Tests  103.  322.  413.  323.  422.  423.  433.  442.  443.  513.  613.  623.  202.  313)/14 
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Avionics  Technician  (AV) 

The  AV  school  administers  knowledge  and  performance  tests,  and  labs.  In 
December,  1991,  the  AV  school  renumbered  and  made  extensive  changes  to  several 
of  the  course  labs  and  tests.  At  that  time,  the  instructors  changed  the  AV 
school  course  number  from  61  to  60.  In  the  following  sections,  we  present  the 
variable  descriptions  and  results  for  Course  61  only,  because  no  cases  were 
available  for  Course  60. 

Description  of  Variables.  The  school  measures  all  tests  on  a  0-100  scale 
and  requires  a  minimum  passing  score  of  70  for  the  course  tests  (knowledge, 
performance,  and  comprehensive).  Students  who  fail  a  test  appear  before  the 
Academic  Review  Board  (ARB).  The  school  then  decides  upon  the  student’s  status. 
ARB  often  permits  such  students  to  retest.  The  instructor  assigns  the  minimum 
passing  score  (70)  to  students  who  pass  the  retest,  and  the  school  uses  that 
minimum  score  in  calculating  the  final  school  grade  (FSG).  The  school  sends 
students  who  fail  the  retest  back  to  ARB  for  a  second  review.  RGI  entered  only 
the  initial  test  scores  for  each  examinee,  because  the  school  assigns  a  score  of 
70  for  all  successful  retests.  The  student  action  code  variable  provides  the 
graduation  status  of  the  student.  The  school  assigned  an  action  code  to  those 
students  who  did  not  complete  the  course.  See  Appendix  B  for  descriptions  of  the 
various  disenrollment  categories. 

Within  Course  61,  the  AV  school  altered,  added,  or  deleted  tests  from  its 
curriculum.  The  following  describes  the  nature  of  each  change  and  its  date  of 

occurrence. 
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In  February  of  1991,  the  AV  school  proposed  major  content  changes  for  all 
Part  4  knowledge  and  comprehensive  tests.  From  February  to  August  of  1991,  the 
^  school  piloted  the  revised  tests  in  classes  90820,  90821,  91010,  91011,  91140, 

I  91141,  91280.  and  91281.  The  new  curriculum  applies  to  all  students  registered 

in  the  course  as  of  April  15,  1991  (Julian  date  91105)  in  addition  to  the  pilot 

classes  listed  above. 

In  addition  to  changing  the  content  of  the  existing  Unit  4  tests,  the 
school  introduced  Knowledge  Test  445  into  the  program.  Only  students  in  the 
pilot  classes  noted  previously  and  students  registered  in  the  course  as  of  91105 
received  Knowledge  Test  445. 

On  June  27,  1991  (Julian  date  91178),  the  school  changed  the  scoring  method 
of  the  course  labs  from  a  percentage  score  to  a  satisfactory/unsatisfactory 
I  grade.  The  school  kept  the  minimum  passing  score  for  each  lab  at  70%.  For 

I  students  who  failed  a  lab,  the  school  retained  its  policy  of  retesting  them  until 

they  passed  the  lab. 

Until  June  28,  1991  (Julian  date  91178),  the  school  included  scores  from 
labs  as  well  as  knowledge,  performance,  and  comprehensive  tests  to  determine  the 
final  school  grade  (FSG)  for  Course  #61.  However,  as  of  this  date,  the  labs  did 
not  contribute  towards  the  FSG.  (See  Appendix  E  for  computation  of  the  FSG.) 

Table  24  presents  the  variable  numbers  and  a  description  of  the  variables 
used  in  the  AV  school  Course  61  data  analysis.  In  parentheses,  following  each 
description,  is  the  name  assigned  to  each  variable. 

1 

1 
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Table  24 


nc<;cnr!ton<  of  AV  School  Variables  •  Course  Number  M 


Test  No.  Description  (Name-Type  of  Test^* 

111  Knoiirlcdgc  Test  General  Math:  Fractions,  basic  algebra,  equations,  signed  numbere.  linear,  tng.,  vector  algebra,  atomic 
structure  (General  Math-K) 

109  Lab:  Intro  to  DC  circuit  analv-sis  &  series  DC  circuit  (Intro/DC  Circuit-L^ 

121  Knowledge  Test  DC  Theory:  multimeters,  DC  circuit  analysis,  electrostatics,  batteries  (DC  Circuiis-K) 

119  Lab:  Parallel  circuits  (Parallel  Circuits-L) 

122  Knowledge  Test  DC  iheoty'*.  Series  circuit  analysis,  parallel  circuits  (Scries/Parallcl  Circuits-K) 

129  Lab:  Scries  parallel  circuits.  DC  troubleshooting  (T/S)  (Serics/Parallcl  DC  T/S-L) 

139  Lab:  \'oltagc  dividers  (\  oltagc  Dividcis-Ll 

124  Performance  Test  T.’S  DC  circuits  iT.  S  DC  Circuus-P) 

123  Comprehensive  Test  DC  Theory  (Comprehensive  1) 

149  Lab:  Application  of  test  equipment  (Test  Equipment-L'l 

159  Lab:  Resistive  capacitive  (RC)  circuit  analvsis  (R/C  Circuit  Analysis-L) 

1^21  Knowledge  Test  AC  concepts,  sine  wave  analysis,  capacitators  &  capacitance,  digital  multimeters.  RC  circuit  analysis, 
oscilloscopes  (OSC)  (AC  Concepts/RC  Circuit/OSC-K) 

169  Lab:  Resistive  inductive  (RL)  circuit  analysis  &  transformers  (R/L  Circuiis.Transf.-L) 

132  Knowledge  Test  Inductors  «S:  inductance.  RL  circuit  analysis,  transformers  (Inductancc-K) 

179  Lab:  Series  &  parallel  resonant  circuits  (Scries/Parallel  Circuiis-L) 

133  Performance  Test  AC:  Application  of  test  equipment,  transformers.  RL  &  RC  circuit  analysis,  scries  reactive  circuits.  RC  & 
RL  signal  filters  (RC/RL  Circ..Filt.-P) 

100  Comprehensive  Test  AC  Theory:  Content  from  lab  #149  to  test  #133  plus  inductivc/capacitivc/resislivc  analysis 

(Comprchensive2) 

219  Lab:  Amplifier  biasing,  transistor  amps  (Amplifiers-L) 

2\\  Knowledge  Test:  Semiconductor  Devices:  Positivc/negaiivc  junctions,  amps/iransistor  thcory»  amp  biasing,  clccrostatic 
discharge  (Semiconductors-K) 

229  Lab:  Limiters  and  clampers  (Limiters  &  Qampers-L) 

221  Knowledge  Test  Solid  State  Devices/Electric  Circuits:  Amp  gain  &  decibels  feedback  amps,  lumters  &  clampers,  classification 
&  coupling,  OSCs  (Solid  State-K) 
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Vablc  24  <Conunuedt 


Tcsi  No.  Dcscnpnon  iNamc-Tvpe  oi  Tcsi)^ 

239  Ub:  OSCs  &  voltage  regulators  (OSa  &  Voltage  Rcgulaion-L) 

241  Knowtedge  Test  Power  Supplies /Nav-a I  Aviation  Maint.  Program:  OSCs.  voltage  regulators,  rectifiers  &  filters,  audio  amps, 

power  supply,  maini.  data  systems  ^P^^T.  Supplies/Naval  Av.  Maini.-K) 

249  Sig.  tracing  AM  receivers  (AM  Receivers-U 

251  KnowicdgcTest  AM  Communication  Theory.  Rad io-frequency(RF)  comm.,  block  diag/tracing/siganalysis/injeciing  AM  Revr. 

transmitter  (Xmtr)  &  RX  fundamentals,  sig  tracing  a  superheterodyne  RX  (AM  Communication  Theory-K) 

259  Lab:  AM  TX.  RF  amp  <S:  1st  RX  mixer  (AM  Xmtr.  RF  Amp.  &  Isi  Revr  Mixer-L) 

269  Ub:  1st  intermediate  freq.  (IF)  amp  &  2nd  RX  mixer,  detector  &  preamp,  audio  output  amp  (1st  IF  Amp.  &  2nd  Re\T 

Mixer-L) 


:"9 

253 

252 

:S9 

299 

254 

209 

255 
200 

309 

319 

321 

329 

339 

331 


Ub:  T/S:  RX  (T/S;  Rcvt-L) 

Performance  Test  T/S;  RX  (T,  S:  Re\T-P) 

Knowledge  Test  AM  TX:  RF  &  IF  amps,  isi  &  2nd  RX  mixer,  detector-preamp,  audio  output  amp  (AM  Xmtr-K) 

Ub:  Low  &  high  freq.  OSC.  Isi  &  2nd  transmitter  mixer  &  OSC.  freq.  amp,  doubler  (OSC.  Xmir-L) 

Ub;  Modulator  (MOD)  &  RF  power  amp,  beat  freq.  OSC.  RX  power  supply  &  voltage  regulator  (Power.  Volt  reg.-L) 

Knowledge  Test  High  &  low  freq.  OSC,  1st  TX  mixer,  freq.  doubler/amp,  beat  freq.  OSC,  MOD  &  RF  power  amp,  TX  power 
supply  &  voltage  regulator  (OSC.  Power-K) 

Ub:  Troubleshooting:  AM  &  FM  TX  (Trouble/S-L) 

Performance  Test  AM  TX  (AM  Xmtr-P) 

Comprehensive  Test  Material  from  Ub  219  to  Test  255  plus  FM  iheory/MOD/deteciion/auto  freq.  control,  transmission  lines, 
antenna  (ant).  T/S  FM  TX.  radio  wave  propagation  (Comprehensive  3) 

Lab:  Integrators  &  differentiators  (Integrators  &  Differentrs-L) 

Ub:  Blocking  OSCs  &  multivibrators  (Blocking  OSCs  &  Multivibrators-L) 

Knowledge  Test  Eleciromcchanical/solid  state  devices,  synchros,  servosystems,  rcsolvcrs/acceleromciers,  blocking  OSCs,  gyros, 
integrators  &  diffmttrs.  motors,  multivibrators  (Elec.  Mech/Solid  State-K) 

Lab:  Sweep  generators  (Sweep  Generators-L) 

Lab:  Switching  power  supplies  (Switching  Power  Supplics-L) 

Knowledge  Test  Power  Supplies:  Sweep  generator,  step  &  level  counters,  diffmtl/opnnl  amps,  voltage  multipliers, 
multiphasc/switching  power  supplies,  integrated  circuits  (Power  Supplics-K) 


®Key  to  type  of  test:  K  -  Knowledge 
L-Ub 

P  -  Performance 
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Table  24  {Conimucd) 


Test  No.  Descnpiion  (Namc-TNpc  of  Tcsi'l 


349  Lab:  Logic  gates  (Logic  Gaics-L) 

J41  Knowtedgc  Test  Number  s>-stems.  logic  gales.  Hip-Hops  (Logic  Gates-KI 


431 


441 


In  Februarv-  of  1991.  the  school  proposed  major  content  ^ 

hf  the  coutse  as  of  April  15. 1991  (Julian:  91105)  in  addition  to  the  pilot  classes  listed 

*^“®*'**^  ^***'  Mnrr  i«  Piw  Maintenance  instruction  manual,  interconnection  wiring  diagrams 

(Only  if  Registemd  Before  91105  and  NOT  ™„ble  units,  radar  fundamentals,  block  diagram 

&  installation/  inspections.  ^Ttem'^o^rion  checks  isolating  malfunctioning  units  in 

«R  /MOD/RX/  w.n, 

s«.em;  T/S  the  fundamentals,  maintenance  instruction  manual.  ASR.  signal  tracing 

B„o.  .nos ...  NOT PRO. 

£.4  p-"= -  -y”-  -‘-“y « ■ 

tf  Rcgtaered  After  91104  or  in  Pilo.  das.)  Block  diag  analysis  ant/  transmitter,  signal  tracing  ant.  T/S  the  ASR  (Pwr. 
Supply/Blk.  Diag.  Aniys-K) 

'^“®***^  n,,nc  „I  wnrr  t- Piln*  riMB3  3-Phase  txwcr  supply/over-load  protection  circuit,  control  circuits. 

iSTir'  »  i  PIR.  a-)  CN... 

c^es.  intermediate  bench  setup,  magnetrons  (3-Phase  Pwr.  Supply/Circ.  Anlys.-K) 

Lab:  T/S:  Ant.  TX.  &  RX  (T/S  Ant,  Xmtr  &  Revr-L) 

before  91105  and  NOT  in  Pilot  Class)  Short-slot  hybrid  mUer/  crystal  current  meter  network,  klystrons. 

Hybrid  Mixcr-K) 

in  ihc  pilot  classes  noted  previously  and  students  rcgtstcrcd  m  the  course  as  oi  ^ 

K«,^Test  (Onlyifn^istered.ftcr911MorinPilotClass)  Circuit  analysis  lOL  circuit  analys/microwave  ant,  T/S:  RX 

ant,  alternate  devices  (Circ.  Anlys.,  Rcvr  Circuit-K) 


442 


459 


443 


445 


444 


Petfonnanee  Test  T/S:  special  circuits  (part  3  &  4  lab  material)  <T/S  Spec.  CircuiU-P) 


‘Key  to  type  of  test:  K  -  Knowledge 
L  -  Lab 

P  -  Performance 
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Fable  24  (Continued) 


Test  No.  Description  ( Name-Ts-pc  of  Test')® 


(oSytf*fe^^Before  91105  and  NOT  in  Pilot  CU«)  Covxrs  material  from  pre-  91105  tests  431.  441.  442.  &  443 

After  91104  or  in  Pilot  Class)  Covers  material  from  post-  91104  tests  431.  441.  442.  443.  &  445. 

(Comprehensive  4) 

509  Lab:  Clocks /counters,  adders/  subtracters,  registers  (Clocks/Couniers.  Adders-L) 

519  Lab:  Multipliers  ^Multipliers-L) 

512  Performance  Test  Digital  circuits  (Digital  Circuits-P) 

511  Knowledge  Test  Registers,  multipliers/  dividers,  elocks/counters.  addeis/subtractors  (Registers/Multipliers/Dividers-K) 

529  Lab:  Signals  (control /address,  timing/data,  memory),  encoders  &  decoders  (Signals.  Encoders.  &  Decoders-L) 

522  Performance  Test  Content  from  lab  #509  to  lab  #529  (Multipliers  •  Subtractors/Signals-P) 


539  Lab:  Intro  to  computer  coding  (Computer  Coding-L) 

As  of  Julv  1  1990  (Julian  date  90183').  the  school  deleted  Comprehensive  Test  500  and  administered  Knowledge  Test  521  at  the  end  of  Part 
The  school  also  revised  Knowledee  Test  521  to  include  material  covered  the  last  week  of  Part  5.  Students  entering  the  course  before 
ihis  date  took  Knowledge  Test  521  dunng  the  24th  week  of  class,  then  they  look  Comprehensive  Test  500  one  week  later.  Stud'n's 
enrolling  after  this  date  did  not  take  Comprehensive  Test  500  (Comprehensive  Test  500  deleted  from  analysis^nly  5  cases).  Instead,  they 
took  ihc  revised  Knowledge  Test  521  in  its  place,  during  the  25th  week. 


521  Knowledge  Test  .  ,  #  ^  *  _ . 

(Only  if  Registered  Before  90183)  Computer  basics,  memory  devices,  microprocessors,  uming  signal  aniys,  data  control. 

adders/memorv,  in-out  interface /devices,  cncoders/dccoders  (Comp.  Basics/Electrostaiic  Dish.-K) 

(Only  if  Rcgistcfd  After  90182)  covers  material  noted  in  test  521  above,  plus  electrostatic  discharge  &  programming  (Comp. 
Basics/Elcctrosiatic  Dish.-K) 

500  Comprebenave  Test  (Only  if  Registered  Before  90183)  Covers  Part  5  material:  Computer  Fundamentals:  Computer  theory, 
processing  circuits,  programming  (Comprehensive  5) 

601  Knowledge  Test  Corrosion,  preventive  maim.,  avionics  equip,  repair/treatment,  equip,  cleaning,  measures  for  elect, 

bonding/grounding,  emergency  procedures  (Corrosion.  Prev.  Maint.-K) 


609  Lab:  Soldering  (Soldering-L) 

619  Lib:  Wire  &  connector  repair,  coaxial  repair  (Wire  Repair-L) 

611  PcifonMnee  Test  Soldering,  wire  connector/  coaxial  repair  (Wire  Repair-P) 

600  Knowiedige  Test  Infrared/laser  principles,  fiber  optics,  cyrogcnics.  automatic  test  equipment,  theater  of  nuclear  warfare 
(Infrared/Laser  Prindples-K) 

7QQ  Knowledge  Test  Basic  integrated  weapons,  communication/navigation/  fire  control/tactical  radar  subsyst.,  electronic  counter 

measures,  anti-sub  warfare,  electromagnetic  interference  awareness  (Basic  Integrated  Wcapons-K) 


^Key  to  type  of  test:  K  -  Knowledge 
L-Lab 

P  -  Performance 
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sample.  The  initial  sample  consisted  of  453  students  who  began  training 
between  March  1990  and  November  1991.  Because  the  AV  school  introduced  a  new 
version  of  Part  4  and  beginning  June  28.  1991  eliminated  labs  from  the  Final 
school  Grade  (FSG)  computation,  we  divided  the  school  within  Course  61  into  3 
groups.  Group  1  Included  those  students  who  received  the  old  version  of  Part  4 
(n=225).  Group  2a  included  those  students  who  received  the  new  version  of  Part 
4  and  whose  FSG  included  labs  (n-SB).  Group  2b  included  students  who  received 
the  new  version  of  Part  4  and  whose  FSG  did  not  include  labs  (n-BS).  Sample 
sizes  for  each  group  include  only  those  students  with  complete  data. 

We  excluded  comprehensive  test  500  from  the  analyses  for  all  three  groups 
because  there  were  only  5  cases  for  that  variable.  We  also  excluded  all  labs 
from  the  factor  analyses.  Many  of  the  labs  had  no  variance,  and  an  initial 
factor  analysis  procedure  could  not  extract  any  factors  using  the  remaining  labs. 

Results  and  Discussion.  We  conducted  an  analysis  to  determine  if  there 
were  differences  in  Group  1  and  2  (2a  and  2b  combined)  due  to  the  Introduction 


of  a  new  version  of  Part  4.  We  conducted  a  multivariate  analysis  of  variance 
(HANOVA)  to  test  the  covariance  matrices  of  the  test  scores:  A  test  for 
homogeneity  of  covariance  matrices  revealed  that  the  two  groups  had  significantly 
different  covariance  matrices  (5-1.70,  pc.OOl).  The  test  means  also  differed 
significantly  across  the  two  groups  (5-3.75,  pc.OOl).  Univariate  f-tests 
indicated  that  20  of  the  34  variables  differed  between  group  at  the  p<.05  level. 

To  test  whether  ability  level  accounted  for  these  differences  between  the 
two  groups  of  students.  We  conducted  a  multivariate  analysis  of  covariance 
(HANCOVA)  using  the  students'  pre-enlistment  ASVAB  scores  as  covariates.  After 
parti  ailing  out  the  variance  due  to  ASVAB  scores,  the  test  for  homogeneity  of 
covaiances  matrices  revealed  different  matrices  for  the  two  groups 
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(F«1.48,  p<.001).  Means  also  differed  across  the  two  groups  after  accounting  for 
variance  due  to  ASVAB  scores  (F=3.20,  p<.001).  These  results  suggest  that  the 
noted  differences  between  the  two  groups  are  due  to  something  other  than 
differences  in  abilty,  and  they  indicate  that  the  two  samples  come  from  two 
distinct  populations. 

To  further  evaluate  the  conclusion  of  differences  between  groups,  we 
computed  effect  sizes  (ES)  as  measures  of  the  pratical  significance  of  the 
differences  between  the  unadjusted  means  across  groups.  Cohen  and  Cohen  (1977) 
state  that  .2  represents  a  small  effect  size,  .5  is  medium,  and  .8  is  large.  For 
the  total  of  total  of  34  comparisons  between  the  AV  groups,  we  found  17  ESs 
within  the  .00  to  .19  range  and  17  ESs  within  the  .20  to  .49  range.  These 
results  show  that  100%  of  the  ESs  fall  below  a  value  of  .50.  Thus,  while  the 
MANOVA  tests  indicate  that  the  groups  differ  from  one  another  in  terms  of 
statistical  significance,  the  ESs  suggest  small  magnitudes  of  the  differences, 
in  terms  of  pratical  significance. 

In  addition  to  testing  for  differences  due  to  the  introduction  of  a  new 
version  of  Part  4,  we  also  tested  for  differences  due  to  including  versus 
excluding  labs  from  the  FSG.  We  compared  Groups  2a  and  2b  using  a  multivariate 
analysis  of  variance  (MANOVA)  to  test  the  covariance  matrices  of  the  test  scores. 
A  test  for  homogeneity  of  covariance  matrices  revealed  that  the  two  groups  had 
significantly  different  covariance  matrices  (F=1.32,  pc.OOl).  The  test  means  did 
not  differ  across  the  two  groups  (F=1.32,  p=.136). 

To  test  whether  ability  level  accounted  for  these  differences  between  the 
two  groups  of  students,  we  conducted  a  multivariate  analysis  of  covariance 
(MANCOVA)  using  the  students’  pre-enlistment  ASVAB  scores  as  covariates.  After 
partialling  out  the  variance  due  to  ASVAB  scores,  the  test  for  homogeneity  of 
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covaiances  matrices  revealed  different  matrices  for  the  two  groups  (F-l.lB. 
p<.001).  Means  did  not  differ  across  the  two  groups  after  accounting  for 
variance  due  to  ASVAB  scores  (F=1.37.  p=.106).  However,  because  of  the 
significant  differences  found  between  the  covariance  matrices,  we  treated  the 
groups  separately.  These  results  suggest  that  the  noted  differences  between  the 
two  groups  is  due  to  something  other  than  differences  in  abilty. 

Based  on  the  differences  due  to  the  new  version  of  Part  4  and  the  two 
methods  of  computing  the  FSG,  we  analyzed  each  of  the  three  groups  separately. 
We  present  the  results  for  Group  1  followed  by  the  results  for  Groups  2a  and  2b. 

Table  25  presents  the  descriptive  statistics  for  the  variables  in  Group  1. 
We  conducted  a  factor  analysis  on  the  set  of  knowledge,  performance,  and 
comprehensive  test  variables.  The  scree  test  and  factor  loadings  indicated  a 
two-factor  solution.  Table  26  shows  the  communal ities,  factor  loadings,  and 
factor  score  coefficients.  Based  on  the  results  of  the  factor  analysis,  we 
labeled  the  first  factor  Advanced  Avionics  and  the  second  factor  Basic  Avionics. 

To  examine  whether  an  Advanced  Avionics  versus  a  Basic  Avionics  distinction 
could  explain  aspects  of  school  performance,  we  created  a  composite  variable 
based  on  the  sum  of  all  the  Advanced  Avionics  tests  (Maint.  Instruction  Manual 
test  to  Basic  Intergrated  Weapons  test)  and  a  composite  variable  based  on  the  sum 
of  all  the  Basic  Avionics  tests  (General  Math  test  to  AC  Theory  test). 
Additional  composites  included  the  Suml  and  Sum2  composites  which  took  a  simple 
average  of  all  the  tests  that  loaded  .30  or  greater  on  either  Factor  1  or  Factor 
2,  respectively.  We  also  calculated  a  composite  based  on  all  Knowledge  tests 
(comprehensive  tests  included),  a  composite  based  on  all  Performance  tests,  and 
a  composite  based  on  all  Labs.  Because  the  AV  school  computed  the  FSG  using  a 
70  as  the  minimum  score,  we  calculated  a  second  final  school  grade  (FSG)  using 
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Tabic  25 


Descnrtivc  Statistics  for  AV  Variables-Group_T 
Variable  J1 


Mean 


Std  Dev  Skewness  Kunosis  Minimum  Maximum 


General  Math-K 

Iniro/DC  Circuii-L 

DC  Circuits-K 

Parallel  Circuiis-L 

Scries /Parallel  Circuiis-K 

Scries /Parallel  DC  T/S-L 

Voltage  Diriders-L 

T/S  DC  Circuiis-P 

Comprehensive  1 

Test  Equipment-L 

R/C  Circuit  Analysis-L 

AC  Concepis/RC  Circuii/OSC-K 

R/L  Circuits.  Transf.-L 

Inductancc-K 

Sencs/Parallel  Circuits-L 

RC/RL  Circ.  Filt.-P 

Comprehensive  2 

Ampiificrs-L 

Semiconduciors-K 

Limiters  &  Clampcis-L 

Solid  Siatc*K 

OSCs  &  Voltage  Regulators-L 

Pv»T.  Supplies/Naval  Av.  Maini.-K 

am  Receive rs-L 

am  Communication  Theory-K 

am  Xmir.  RF  Amp,  &  1st  Revr  Mixer-L 

1st  IF  Amp.  &  2nd  Revr  Mixer-L 

T/S:  Revr-L 

T/S:  Revr-P 

am  Xmtr-K 

OSC  Xmtr-L 

Pov^xr.  Volt  rcg.-L 

OSC.  Power-K 

Trouble /S-L 

am  Xmir-P 

Comprehensive  3 

Integrators  &  Differentrs-L 

Blocking  OSCs  &  Multivibrators-L 

ElecMech/Solid  Statc-K 

Sweep  Generators-L 

Power  Supplies-K 

Logic  Gates-L 

Logic  Gates-K 

Maint.  Instruaion  Manual-K 
Pwr.SuppIy/Blk.DiagAnlys-K 
3-Phase  Pwr.SuppIy/CircAnlys.-K 
T/S  Ant,  Xmtr,  &  Revr-L 
Short-slot  Hybrid  Mixer-K 
T/S  Spec.  Circuits-P 
Oamprehensive  4 
Oocks/Counters  A<lders-L 
Multipliers-L 
Digital  Circuits-P 
Registcrs/Multipliers/Dividers-K 
Signals,Encodeis,&  Decoders-L 


234 

87.179 

12.796 

2.079 

3730 

100.00 

234 

93.189 

7.305 

-2.458 

7.124 

53.20 

100.00 

234 

84.797 

9330 

-0379 

-0.149 

55.00 

100.00 

233 

93.240 

5.333 

-1.133 

1.103 

71.80 

100.00 

233 

84.421 

12.245 

-1.194 

1.905 

3730 

100.00 

231 

93.402 

7309 

-3.430 

15.461 

45.60 

100.00 

230 

93.039 

9339 

-♦.159 

27306 

10.00 

100.00 

231 

95.311 

7.058 

-2.222 

6.836 

52.00 

100.00 

230 

82.484 

10.615 

-0.356 

-0381 

52.00 

100.00 

231 

95.750 

8.736 

-5.049 

40.829 

10.00 

100.00 

231 

93.010 

9.205 

-2.163 

5.983 

46.20 

100.00 

231 

82.076 

12.219 

-0389 

-0322 

48.00 

100.00 

230 

95i71 

5.139 

-4.683 

35.287 

47.70 

100.00 

230 

81.760 

11.876 

-0330 

-0.264 

48.60 

100.00 

228 

92.729 

9.689 

-2.362 

5.877 

50.00 

100.00 

228 

97.693 

4.782 

4.301 

26.345 

58.00 

100.00 

228 

79.044 

9.793 

0.055 

-0.813 

56.00 

100.00 

228 

98i21 

5.334 

-7.911 

73.395 

41.60 

100.00 

228 

85.417 

9.421 

-0.654 

0.152 

55.00 

100.00 

228 

98.907 

6.154 

-13.446 

194.087 

10.00 

100.00 

228 

85.876 

10.199 

-1.722 

7.120 

20.00 

100.00 

228 

98.984 

1.774 

-2326 

7.187 

90.00 

100.00 

228 

84.058 

10.212 

4).992 

3.707 

25.70 

100.00 

228 

97.480 

13.728 

-6.357 

39375 

10.00 

100.00 

228 

86.009 

8.863 

-0.697 

0.271 

5730 

100.00 

228 

97.470 

3.663 

-1.673 

2.913 

80.00 

100.00 

228 

97.680 

4.089 

-3.136 

15.047 

6730 

100.00 

228 

94.727 

8.063 

-2.305 

6.603 

48.70 

100.00 

228 

94.936 

9.049 

-3.712 

19.251 

27.00 

100.00 

228 

80.976 

10.987 

-0.650 

1.065 

30.00 

100.00 

228 

98.708 

4.739 

-10.401 

132.072 

3630 

100.00 

228 

99.421 

1362 

-3.974 

18320 

89.00 

100.00 

228 

85.954 

8.074 

-0306 

-0.070 

6230 

100.00 

228 

97.432 

7.812 

-6.991 

69.808 

10.00 

100.00 

228 

•  %.872 

6.867 

-6.903 

72.711 

19.00 

100.00 

228 

82.219 

8391 

-0.380 

0.399 

54.00 

100.00 

228 

99.211 

8.411 

-10.606 

111.464 

10.00 

100.00 

228 

98.033 

12.117 

-6.166 

37.274 

10.00 

100.00 

228 

80.998 

9.705 

4).310 

-0.280 

5230 

100.00 

228 

99.189 

8.415 

-10382 

111.110 

10.00 

100.00 

228 

84.035 

8.112 

4).335 

0.331 

55.00 

100.00 

227 

99359 

4.692 

-10382 

110.964 

49.90 

100.00 

228 

88.918 

7.988 

-1.015 

1.246 

5430 

100.00 

228 

88.759 

11.639 

-3.424 

14.974 

26.70 

100.00 

227 

84.220 

9.181 

-0.842 

0.944 

45.00 

100.00 

227 

86.991 

7.962 

-1.292 

3.890 

4190 

100.00 

227 

99.814 

1.613 

-8.627 

73312 

8530 

100.00 

227 

85.253 

10.897 

-1.004 

1357 

4230 

100.00 

227 

94,876 

5.190 

-1.473 

2.708 

70.00 

100.00 

227 

89314 

7.974 

-2.021 

7358 

4730 

100.00 

227 

96.819 

15353 

-5.428 

27345 

10.00 

100.00 

227 

96868 

15368 

-5.422 

27.792 

10.00 

100.00 

227 

97.899 

4364 

4.083 

24322 

60.00 

100.00 

227 

84.722 

9.664 

-1.469 

5.130 

28.60 

100.00 

227 

99.733 

0.903 

-3.415 

10341 

96.00 

100.00 

Table  25  (  Conunucd) 


Variable 

Muliiphcrs-Subtraciore/Signals-K 

Computer  Coding-L 

Comp.  Basics/Electrostatic  Disch.-K 

Corrosion.  Prcv.  Maini.-K 

Soldering-K 

Wire  Repair-L 

Wire  Rcpair-P 

Infrared /Laser  Principics-K 

Basic  Integrated  Weapons-K 


Sid  Dev 


Skewness 


Kurtosis 


Minimum  Ma»mum 


Table  26 


F3c.o^  I  and  Fac.or  .Score  Cocfficien.s  for  AV  Variables  ■  Grout 


Communaiit>' 


Factor  Loadings  _ 
Factor  1:  Factor!: 

Adv  Avionics  Bsc  Avionics 


Factor  Score  Coefficients  _ 
Factor  1:  Factor  2: 

Adv  Avionics  Bsc  Avionics 


General  Math-K 

DC  Circuiis-K 

Scries/Parallel  Circuits-K 

T/S  DC  Circuiis-P 

Comprehensive  1 

AC  Concepts/ RC  Circuit/OSC-K 

Inductance-K 

RC/RL  Circ.  Filt.-P 

Comprehensive  2 

Sc  miconductors-K 

Solid  State-K 

Pwr.  Suppiies/Naval  Av.  Maint.-K 

am  Communication  Theory-K 

T/S:  Rcvt-L 

am  Xmtr-K 

OSC.Powcr-K 

am  Xmtr-P 

Comprehensive  3 

ElecMech /Solid  State-K 

Power  Supplies-K 

Maint.  Instruction  Manual-K 

Pwr.Supply/Blk-Diagj\nlys-K 

3-Phase  pir.Supply/Circ-Anlys.-K 

Short-slot  Hybrid  Mixer-K 

T/S  Spec.  Circuits-P 

Coomprchcnsivc  4 

Digiul  Circuits-P 

Rcgistcrs/Multipliers/Dividers-K 
Multiplicrs-Subtractors/Signals-K 
Comp.  Basics/Electrostatic  Disch.-K 
Corrosion.  Prcv.  Maint.-K 
Wire  Rcpair-P 
Infrarcd/Lascr  Prtndplcs-K 
Basic  Integrated  Wcapons-K 

Note,  n  =  225 


49 


the  initial  test  scores,  and  listed  both  as  composite  variables.  Table  27  shows 
the  descriptive  statistics  for  the  composite  variables. 

Next,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  28  presents  the  composite  variable  intercorrelations.  Because 
FSG  and  FSG2  correlate  .98,  this  offers  no  support  for  using  initial  test  scores 

to  compute  FSG. 

Advanced  Avionics  correlates  highest  (.95)  with  Factorl,  followed  by  Suml 
(.83).  Similiarly,  Basic  Avionics  correlates  highest  (.91)  with  Factor2, 
followed  closely  by  Sum2  (.85).  These  findings  support  using  the  logical 
combinations  of  tests  to  assess  the  dimensions  reflected  in  the  factor  analyses. 

Finally,  FSG  corrlelates  .77  with  the  Advanced  Avionics  and  .87  with  Basic 
Avionics.  Considering  the  moderate  correlations  of  FSG  with  Factors  1  and  2  (.72 
and  .75),  these  findings  do  not  support  FSG  as  a  unidimensional  criterion. 
Instead,  these  findings  suggest  using  the  composites  Advanced  Avionics  and  Basic 
Avionics. 

Results  for  Groups  2a  and  2b.  Table  29  presents  the  descriptive 
statistics  for  Group  2a  and  Table  30  presents  the  descriptive  statistics  for 
Group  2b.  Because  of  the  small  sample  sizes  in  both  groups,  a  factor  analysis 
procedure  could  not  extract  factors  for  either  group.  As  a  result,  we  computed 
composite  variables  based  on  the  same  composite  variables  computed  for  Group  1. 
Table  31  and  Table  32  present  the  descriptive  statistics  for  the  composite 
variables  for  Group  2a  and  Group  2b,  respectively. 

Next,  we  examined  the  patterns  of  correlations  among  the  composite 
variables  for  each  group.  Table  33  presents  the  correlation  matrix  for  both 
Groups  2a  and  2b.  For  both  groups,  FSG  and  FSG2  correlate  .97  to  .99  which 
offers  no  support  for  using  initial  test  scores  to  compute  FSG.  Also,  for  Group 
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Table  ;* 


Desenptive  Statisiics  for  AV  Composite  Variables  -  Ciroun  1 


Composite 

Rel^ 

.Mean 

Sid  pev 

Skewness 

Kurtosis 

.Minimum 

.Maximum 

Suml“ 

226 

.928 

5.772 

0.121 

-0.712 

72.33 

97.47 

Sum2^ 

227 

.927 

85.019 

6.253 

0.341 

-0.687 

69.69 

98.63 

FSG*^ 

234 

NVA 

89.669 

4,865 

•1.838 

9.048 

60.60 

98.60 

FSG2® 

225 

.894 

88.260 

4555 

0,123 

43.756 

7754 

98.15 

Knowledge  “ 

225 

.936 

85.158 

5.844 

0.165 

-0.772 

72.19 

97.97 

Performance* 

226 

.281 

96i39 

2.448 

•1.455 

4527 

82.25 

100.00 

Ub'^ 

226 

.861 

97.174 

2.181 

-2.248 

7.323 

83.90 

99,73 

.Adv  Avionics “ 

226 

.817 

87.9fi3 

5.676 

-0.692 

1.068 

64.94 

9859 

Bsc  Avionics*^ 

228 

.884 

84.960 

7.824 

-0.224 

-0.906 

68.20 

99.69 

•  VII  ^viiiiiiMiiaiii  V  CIIIU  illb  U»V.J1UCU  III  IIIC 

"Suml:  Average  of  all  variables  loading  .30  or  greater  on  Factor  1. 

^Sum2;  Average  of  all  variables  loading  .30  or  greater  on  Factor  2. 

^FSG:  Final  School  Grade  computed  by  the  AV'  school  (see  Appendix  E). 

®FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

“Knowledge:  Average  of  all  Knowledge  tests. 

^Performance:  Average  of  all  Performance  tests. 

"Lab:  Average  of  all  Lab  tests 

\Adv  Avionics:  .Average  of  .Maim.  Instructional  Manual  to  Basic  imergraied  Weapons  tests. 
“Bsc  Avionics:  Average  of  General  Math  to  AC  Theory  tests. 


Table  28 

Correlations  Amoung  AV  Comtxjsites  and  Factors  -  Group  1 


Comp 

Factor  1 

Facior2 

Suml® 

Sum2^ 

FSG' 

FSG2“ 

Knowledge® 

Perform^ 

Lab* 

Adv  Avionics*"^ 

Factor2 

.13 

Suml 

.83 

.64 

Sum2 

.62 

.85 

.94 

FSG 

.72 

.75 

.96 

.96 

FSG2 

.79 

.69 

.99 

.96 

.98 

Knowledge 

.78 

.71 

.99 

.97 

.98 

.99 

Perform 

.27 

52 

.49 

55 

58 

58 

.49 

Ub 

.19 

.24 

.26 

.27 

.30 

.30 

.28 

.18 

Adv  Avionics 

.95 

.24 

.86 

.67 

.77 

.83 

.82 

.34 

.23 

Bsc  Avionics^ 

.43 

.91 

,82 

.92 

.87 

.84 

.87 

51 

.30 

54 

Note.  n:=  225 

*Suml:  Average  of  all  variables  loading  .30  or  greater  on  Factor  1. 

^Sum2:  Average  of  all  variables  loading  JO  or  greater  on  Factor  2. 

®FSG:  Final  School  Grade  computed  by  the  AV  school 
^FSG2:  See  Appendix  E  for  computation  of  Final  School  Grade 
•Knowledge:  Average  of  all  Knowledge  tests. 

^Performance:  Average  of  all  Performance  tests. 

^Lab:  Average  of  all  Lab  tests 

^Adv  Avionics:  Average  of  Maint.  Instructional  Manual  to  Basic  Intergrated  Weapons  tests. 
^Bsc  Avionics:  Average  of  General  Math  to  AC  Theory  tests. 
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'I\ibie  2v 


Descnrtivc  Statistics  for  AN'  Variables  -  Group  2a 


\  anabic 

n 

Mean 

Std  Dev 

Skewness 

Kunosis 

.Minimum 

Manraum 

General  Maih-K 

101 

88.218 

13.136 

-1,755 

3.060 

40.00 

100.00 

Intro/DC  Circuii-L 

100 

93.740 

7340 

-2.478 

6.639 

62.40 

100,00 

DC  Circuiis-K 

100 

87.825 

8.293 

-0.798 

0.294 

65.00 

100.00 

Parallel  Circuits-L 

100 

93.699 

5312 

-0.707 

-0303 

80.10 

100.00 

Scrics/Parallel  Circuiis-K 

100 

86.650 

11.148 

-1.014 

0.811 

50.00 

100.00 

Scries/ Parallel  DC  T/S-L 

99 

94.816 

9307 

-6.069 

45.826 

1730 

100.00 

\  oltage  Di^iders-L 

99 

95.385 

5.049 

-1.252 

1.401 

7630 

100,00 

T/S  DC  Circuits-P 

99 

96.030 

6.429 

-1.774 

2.705 

72.00 

100.00 

Comprehensive  1 

99 

86.437 

9.243 

■0.718 

0.470 

57.80 

100.00 

Test  Equipment-L 

99 

97.814 

5.319 

■4A14 

23.622 

62.00 

100.00 

R/C  Circuit  Analv-sis-L 

99 

94.839 

S.700 

-2^64 

6.987 

54.90 

100,00 

AC  Conccpts/RC  Circuit/OSC-K 

98 

84.133 

11.CM2 

-0.697 

-0.058 

5230 

100,00 

R/L  Circuits  Transf.-L 

98 

96.395 

4.762 

-3.095 

13.458 

68.20 

100.00 

Inductance-K 

98 

81.893 

11.935 

-0.543 

-0.214 

48.60 

100.00 

Scrics/Parallel  Circuits-L 

97 

96J98 

5.3« 

-3.240 

13357 

68.00 

100.00 

RC/RL  Circ.  Filt.-P 

97 

98.392 

2.819 

-1,666 

1.788 

88.00 

100.00 

Comprehensive  2 

97 

81.423 

10387 

-0.557 

0.282 

48.00 

100.00 

Amplificrs-L 

95 

99.712 

1.053 

-1.058 

16.395 

94.10 

100.00 

Scr*;conductors-K 

97 

86.010 

11.325 

-1.312 

1.796 

4230 

100.00 

Lir.  .:ers  6:  Clampers-L 

96 

99.724 

0.984 

-3.639 

12.320 

95.00 

100.00 

Solid  State-K 

96 

86.452 

9.973 

4).466 

-0399 

60.00 

100.00 

OSCs  &  Voltage  Rcgulators-L 

96 

99.906 

0.327 

-3.710 

14.496 

98.00 

100.00 

PwT.  Supplies /Naval  Av.  Maini.-K 

95 

85.892 

9.183 

-0.353 

-0.467 

60.00 

100.00 

AM  Rcceivcrs-L 

95 

99.220 

7.602 

-9.747 

95.000 

25.90 

100.00 

AM  Communication  Thcory-K 

95 

86.868 

9.432 

-1.151 

1.486 

5530 

100.00 

AM  Xmir.  RF  Amp.  &  Isi  Revr  Mixer-L 

94 

99.843 

0.733 

-5.343 

30.966 

94.70 

100.00 

1st  IF  Amp.  &  2nd  Revr  Mixer-L 

94 

99.777 

0.941 

^323 

20.209 

95.00 

100.00 

T/S:  Revr-L 

94 

98.006 

6.469 

-3352 

12.004 

70.00 

100.00 

T/S:  Revr-P 

94 

%S00 

8.061 

^.985 

33.995 

36.00 

100.00 

AM  Xmtr-K 

94 

84.681 

9.866 

-0312 

0.286 

55.00 

100.00 

OSC,  Xmir-L 

93 

99.944 

0339 

-9.644 

93.000 

94.80 

100.00 

Power,  Vol  rcg.-L 

93 

99.980 

0.197 

-9.644 

93.000 

98.10 

100.00 

OSC.  Power- K 

93 

87.312 

9.046 

-1.211 

1,824 

55.00 

100.00 

Troubic/S-L 

93 

99.271 

3.701 

-5.189 

25.950 

79.10 

100.00 

AM  Xmtr-P 

93 

97.613 

6.656 

-4.378 

21.484 

60.00 

100.00 

Comprehensive  3 

93 

84.231 

8.760 

-0,184 

-0.478 

62.00 

100.00 

Elec.Mcch /Solid  State-K 

93 

83.548 

9.400 

-0.408 

-0.251 

60.00 

100.00 

Switching  Power  Supplies-L 

93 

98.935 

10.266 

-9.644 

93.000 

1.00 

100.00 

Power  Supplies-K 

93 

85323 

7.899 

-0.356 

-0339 

6730 

100.00 

Logic  Gates-L 

93 

95.161 

20.410 

^.022 

14.488 

10.00 

100.00 

Logic  Gates-K 

93 

90.932 

7301 

-0.960 

0.821 

65.70 

100.00 

Maint.  Instruction  Manual-K 

93 

92.940 

4.806 

-0.316 

-0305 

82.10 

100.00 

Pwr.Supply/BIk.DiagAnlys-K 

93 

86.860 

9.404 

-0.382 

-0.844 

65.00 

100.00 

3-Phasc  Pw.Supply/Circj\nlys.-K 

93 

88377 

10.032 

-2.153 

9.960 

32.10 

100.00 

Short-slot  Hybrid  Mixer-K 

93 

86.842 

9.372 

-0.722 

0.348 

55.00 

100.00 

CircAnlys.,  Revr  Circuit-K 

89 

87.449 

10.020 

-0374 

-0334 

6230 

100.00 

T/S  Spec.  Circuits-P 

93 

95.935 

6306 

-2.101 

4.370 

69.00 

100.00 

Comprehensive  4 

93 

87303 

8.053 

-0326 

-0.216 

6230 

100.00 

Qocks/CountexsAddeis-L 

93 

99.677 

0.823 

-2.322 

3.920 

97.00 

100.00 

Multiplien-L 

93 

99.624 

0.955 

-2.240 

3.270 

97.00 

100.00 

Digital  Circuits-P 

93 

99.194 

2.086 

-2.787 

7.693 

90.00 

100.00 

Registers/Multipliers/Dividers-K 

93 

85.802 

9.970 

-0.696 

0.686 

48.60 

100.00 

Signat$,Encoders,&  Decoders-L 

93 

99.828 

0.816 

-4379 

19.385 

96.00 

100.00 

Multipliers-Subtractois/Signals-K 

93 

98.946 

3.181 

-3.919 

17.480 

80.00 

100.00 

Computer  Coding-L 

93 

99.871 

0.612 

^379 

19385 

97.00 

100.00 

I  able  29  fConiinuea ) 


\anable 

n 

Mean 

Std  Dev 

Skewness 

Kunosis 

.  Minimum 

.Maximum 

Comp,  basics,  bicctrosrauc  Disch.-K 

93 

Ji5.989 

9.217 

-0.462 

-0.792 

65.00 

Corrosion.  Prcv.  Maini.-K 

93 

92.322 

7J24 

-1J52 

3J93 

59.30 

Soldenng-K 

93 

99.828 

0.816 

-4.579 

19.385 

96.00 

Wire  Rcpair-L 

93 

99.871 

0.612 

4J79 

19.385 

97.00 

Wire  Rcpair-P 

93 

95.742 

5.211 

-1.909 

3.911 

75.00 

In  fra  red /Laser  Principles-K 

93 

90.323 

S.634 

-0.832 

-0.189 

68.00 

Basic  Integrated  Wcapons-K 

92 

92.310 

8.723 

-1.183 

0.658 

66.70 

Table  30 

Descriptive  Statistics  for  AV  Variables 

-  GrouD  2b 

V'aiiabie 

j) 

.Mean 

Std  Dev 

Skewness 

Kurtosis 

.Minimum 

.Maximum 

General  .\lath-K 

115 

84.130 

15.232 

-1352 

2,440 

2230 

DC  Circuiis-K 

no 

83.136 

10.060 

-0.704 

0.224 

5230 

Series/ Parallel  Circuiis-K 

110 

84.091 

13331 

-1.101 

0.782 

4230 

T/S  DC  Circuiis-P 

108 

94.139 

8364 

-2.267 

6.105 

54.00 

Comprehensive  1 

108 

83341 

11.743 

-1.218 

3.493 

31.10 

AC  Concepts/ RC  Circuii/OSC-K 

105 

82.071 

13.452 

-1.059 

0.466 

40,00 

Inductance-K 

98 

83.266 

11.248 

-0397 

-0.251 

51.40 

RC/RL  Circ.  -PFill. 

94 

97.011 

5.654 

4.2S7 

26354 

57.00 

Comprehensive  2 

94 

82.234 

9.421 

-0.678 

1.841 

44.00 

Semiconductors-K 

92 

86.646 

8.257 

-0350 

0.159 

6030 

Solid  State-K 

91 

86.749 

9.080 

-0.416 

-0.388 

60.00 

Pwr.  Supplies/Naval  Av.  Maint.-K 

91 

84.645 

9.776 

-0.468 

0.061 

5430 

AM  Communication  Theoiv-K 

89 

88311 

8.383 

-1.362 

2.139 

60.00 

T/S:  Revr-L 

85 

97.941 

3343 

-1376 

0.927 

89.00 

AM  Xmtr-K 

85 

86.324 

8.234 

-0.372 

-0.447 

65.00 

OSC.  Power-K 

85 

88.618 

6352 

-0.260 

-0.650 

75.00 

AM  Xmtr-P 

85 

98.753 

3.128 

-2.459 

4.396 

89.00 

Comprehensive  3 

85 

85.912 

8.477 

-0.792 

0.147 

63.30 

Elcc.Mcch/Solid  State-K 

85 

85.353 

8.601 

-0.665 

0.264 

60.00 

Power  Supplies-K 

85 

86.676 

7.414 

-0.456 

-0.418 

6730 

Maint.  Instruction  Manual-K 

84 

93.025 

5.726 

-1.443 

3.425 

67.90 

Pwr.Supply/Blk.DiagAnlys-K 

84 

86.726 

8.159 

-0309 

-0.423 

70.00 

3-Phasc  PwT.Supply/Circj\nlys.-K 

84 

89.074 

8.264 

-0.863 

0343 

64.30 

Shon-slot  Hybrid  Mixer-K 

84 

85.060 

9.863 

-0.702 

43.036 

60.00 

Circ.Anlys.,  Revr  Circuii-K 

84 

86.410 

9.418 

-0.641 

0.377 

5430 

T/S  Spec.  Circuits-P 

84 

96.083 

7.159 

-3.009 

10.383 

60.00 

Comprehensive  4 

84 

87.887 

8.108 

-1.262 

2.110 

5730 

Digital  Circuits-P 

83 

98354 

2.860 

-2.426 

6.681 

85.00 

Registers/Multipliers/Divideis-K 

83 

87.157 

8.604 

-0.636 

-0.042 

62.90 

Multipliers-Subtiactors/Signals-K 

83 

99.398 

1.814 

-3.127 

9.971 

90.00 

Comp.  Basics/Electrostatic  Disch.-K 

83 

86.235 

8350 

-0.696 

0,387 

6230 

Corrosion,  Prev.  Maint.-K 

83 

92.288 

7.152 

-0.750 

-0.132 

74.10 

Wire  Repair-P 

83 

95.855 

5.469 

-1.921 

4.250 

75.00 

Infrared/Laser  Piinciples-K 

83 

90.795 

7.771 

-0.646 

-0.413 

72.00 

Basic  Integrated  Weapons-K 

83 

90.677 

10,611 

-1.941 

4.256 

5330 

IH 
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Table  31 


Descnr:ive  Siaiisnc?  I'o:  .-W’  Crn'.rosiie  N'anahics  -  Group  2a 


Composite 

r\ 

Rel® 

.Mean 

Sid  Dev 

Skewness 

Kurtosis 

.Minimum 

Maximum 

Suml^ 

93 

.979 

86.947 

6.006 

0.070 

•0.723 

73.66 

98.87 

Sum2^ 

93 

.979 

86.791 

6.498 

-0.139 

-0.660 

72i3 

99.14 

FSG= 

101 

N7A 

90.765 

4.948 

•1.231 

3.402 

69.60 

99.30 

FSG2® 

88 

.972 

89.900 

4.820 

-0.174 

-0.663 

78,01 

99.21 

Knowledge^ 

88 

.982 

87.109 

5.196 

-0.071 

-0.873 

73,80 

99.11 

Performance^ 

93 

,831 

97.335 

2.768 

-3.230 

15,054 

80.63 

100.00 

Lab“ 

93 

.882 

97J69 

2.043 

-1.396 

li71 

91.22 

100.00 

Adv  Avionics^ 

93 

.925 

89,616 

5.094 

•0.355 

-0J17 

76.84 

98.22 

Bsc  Avionics  j 

97 

.962 

86.764 

7.704 

-0-507 

-0.373 

66.79 

99.38 

^  I - - - - - —  wx  WO  wwovxit/wu  111  IIIW 

Suml;  Average  of  all  variables  loading  .30  or  greater  on  Factor  1. 

^Sum2:  Average  of  all  v-ariablcs  loading  .30  or  greater  on  Factor  2. 

^FSG:  Final  School  Grade  computed  by  the  AV'  school 
®FSG2:  See  Appendix  E  for  computation  of  Final  School  Grade 
•Knowledge:  Average  of  all  Knowledge  tests. 

^Performance:  Average  of  all  Performance  tests. 

•'''Lab:  Average  of  all  Lab  tests 

^Adv  Avionics:  Average  of  Maint.  Instructional  Manual  to  Basic  Intergrated  Weapons  tests. 
•^Bsc  Avionics:  Average  of  General  Math  to  AC  Theory  tests. 


Tabic  32 

Descriptive  Statistics  for  AV  Composite  Variables  -  Group  2b 


Composite 

n 

Rel* 

Mean 

Std  Dev 

Skewness 

Kurtosis 

Minimum 

.Maximum 

Suml^ 

83 

.951 

87.635 

4.810 

•0.122 

-0,882  • 

77.17 

96.71 

Sum2^ 

83 

.953 

87J31 

5.355 

-0.061 

-0.938 

76.60 

97.69 

FSG"^ 

115 

N/A 

86359 

11.130 

-3.098 

12321 

2230 

97,30 

FSG2« 

■f 

83 

.940 

88.834 

4.666 

0.207 

-0.818 

77.85 

97.17 

Knowledge^ 

83 

.958 

87357 

4.890 

-0.198 

-0.844 

77.41 

9638 

Performance^ 

83 

-552 

97.295 

2.013 

•1.416 

2.208 

89.38 

99.75 

Adv  Avionics" 

83 

.828 

89.680 

4.214 

-0.281 

-0371 

79.92 

9739 

Bsc  Avionics^ 

94 

.913 

86.045 

7365 

-0310 

-0.433 

65.40 

98.79 

*Thesc  arc  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
Suml:  Average  of  all  variables  loading  .30  or  greater  on  Factor  1. 

®Sum2:  Average  of  all  variables  loading  .30  or  greater  on  Factor  2. 

FSG:  Final  School  Grade  computed  by  the  AV  school  (see  Appendix  E). 

*FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

^Knowledge:  Average  of  all  Knowledge  tests. 

^Performance:  Average  of  all  Performance  tests. 

Adv  Avionics:  Average  of  Maint.  Instructional  Manual  to  Basic  Intergrated  Weapons  tests. 

^Bsc  Avionics:  Average  of  General  Math  to  AC  Theory  tests. 
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Tabic  33 


Correiations  Am^^ng  A\’  Camvo^nc  Varianles  «  Groups  2a  &  2b 


Composiic 

Suml^ 

Suma*^ 

FSG' 

FSG2° 

Knowledge® 

Perform^ 

Lab* 

Adv  Avionics^ 

Bsc  Avionics' 

Suml 

• 

.94 

.97 

.95 

.98 

.52 

n/a 

.84 

.78 

Sum2 

.97 

- 

.98 

.99 

.97 

.56 

n/a 

.67 

.91 

FSG 

.98 

.98 

- 

.98 

.98 

.64 

n/a 

.78 

.86 

FSG2 

.99 

.97 

.99 

- 

.97 

.60 

n/a 

.70 

.90 

Knowledge 

.99 

.98 

.99 

.99 

- 

JO 

n/a 

.80 

.84 

Perform 

.51 

.54 

.60 

i8 

.49 

- 

n/a 

.39 

-56 

Lab 

.46 

.46 

.43 

.45 

.46 

.13 

- 

n/a 

n/a 

Adv  Avionics 

.91 

.81 

.88 

.91 

.89 

.47 

.35 

- 

.50 

Bsc  Avionics 

.88 

.94 

.91 

.88 

.91 

J4 

.46 

.69 

- 

Note.  Cocfficicnis  below  the  diagonal  correspond  to  Group  2a  (n  =  88).  Coefficients  above  the  diagonal 
correspond  to  Group  2b  (n  =  83). 

®Suml:  Average  of  all  vanablcs  loading  .30  or  greater  on  Factor  1. 

^Sum2:  Average  of  all  variables  loading  .30  or  greater  on  Factor  2. 

^FSG:  Final  School  Grade  computed  by  the  AV  school  (see  Appendix  EV 
"FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

"Knowledge:  Average  of  all  Knowledge  tests. 

“P:  rformance:  Average  of  all  Performance  tests. 

-I  j:  Average  of  all  Lab  tests 

^Adv  Avionics:  Average  of  Maint.  Instructional  Manual  to  Basic  Intcrgrated  Weapons  tests. 

•Bsc  Avionics:  Average  of  General  Math  to  AC  Theory  tests. 


Advanced  Avionics  correlates  .88  with  FSG  and  Basic  Avionics  correlates 
.91  with  FSG.  These  findings  support  using  FSG  as  an  indicator  of  performance 
on  Advanced  Avionics  tests  and  Basic  Avionics  tests. 

For  Group  2b,  FSG  and  FSG2  correlate  .98  which  offers  no  support  for  using 
initial  test  scores  to  compute  FSG,  Suml  and  Sum2  correlate  .97  and  .94,  offering 
no  support  for  2  different  factors.  Indeed,  correlations  with  Suml  or  Sum2  with 
FSG,  Knowledge,  Advanced  Avionics,  and  Basic  Avionics  range  from  .81  to  .98.  The 
lowest  correlations  with  Suml  and  Sum2  are  with  the  Performance  and  Lab 
composites  (.46  to  .56).  These  results  support  using  the  Performance  and  Lab 
composites. 

Advanced  Avionics  correlates  .78  to  .88  with  FSG  and  Basic  Avionics 
correlates  .86  to  .91  with  FSG.  These  findings  support  using  FSG  as  an  indicator 
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of  performance  on  Basic  Avionics  tests  but  suggest  using  a  separate  indicator  for 
performance  on  the  Advanced  Avionics  tests. 

Recommendations.  For  Group  1,  in  addition  to  FSG,  use  the  average  of  all 
Advanced  Avionics  tests  and  the  average  of  all  Basic  Avionics  tests.  For  Groups 
2a  and  2b,  in  addition  to  FSG,  use  an  average  of  all  Performance  tests  and  an 
average  of  all  Lab  tests  (except  where  not  available  for  Group  2b). 
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Electrician’s  Hate  (EM) 

Description  of  Variables.  Table  34  presents  the  EM  tests  and  their 
corresponding  descriptions.  The  EM  school  assesses  student  progress  using 
written  knowledge  and  comprehensive  tests.  The  tests  have  a  valid  range  of  0  to 
100.  The  EM  school  separates  the  curriculum  into  3  phases.  Phase  1  covers  the 
first  9  tests,  ending  after  the  first  comprehensive  test.  This  phase  addresses 
EM  basics.  The  next  11  tests  comprise  Phase  2,  ending  with  the  third 
comprehensive  exam.  Phase  3  covers  the  remaining  4  tests.  Phase  2  and  Phase  3 
address  EM  technical  material.  Table  34  points  out  a  change  in  test  order  for 
3  of  the  tests  (Lighting,  Small  Craft,  and  Auxilliary). 

The  EM  school  requires  a  minimum  passing  score  of  75%  for  each  test.  The 
Academic  Review  Board  (ARB)  forms  a  recommendation  as  to  the  status  of  each 
student  who  fails  a  test.  ARB  often  permits  such  students  to  retest.  ARB 
reexamines  students  who  fail  the  retest  and  determines  how  the  student  will 
proceed.  The  instructor  assigns  the  minimum  passing  score  to  those  students  who 
pass  the  retest,  and  the  school  uses  that  minimum  score  in  calculating  the  final 
school  grade  (FSG).  Although  the  instructor  enters  a  score  of  75  for  all 
successful  retests,  for  the  purpose  of  the  present  study,  RGI  entered  all  initial 
test  scores  into  the  database. 

The  EM  school  uses  a  weighted  average  of  the  test  scores  to  determine  the 
final  FSG.  Each  knowledge  test  contributes  4%  and  each  comprehensive  test 
contributes  6%.  The  instructors  do  not  include  the  Math  Pretest  score  (test 
#500)  in  determining  the  FSG. 

Sample.  The  initial  sample  contained  961  students  who  began  training 
between  January  1990  and  October  1991.  We  excluded  students  who  did  not  take  all 
tests  from  some  analyses.  Consequently,  our  final  sample  contained  891  cases. 
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Table  >4 


Descripnons  of  FNf  N'anaMcs 


Test  No.  Descnption  (Name) 


Phase  I 


500 

001 

002 

004 

005 

006 

fX)7 

^'•08 

':'09 


Knowledge  Test  Math  pretest  to  determine  skills  upon  course  entrance  (FSG  docs  not  include  this  test)  (Math  Pretest) 
Knowledge  Test  General  math,  scientific  notation,  trigonometry  (Math) 

Knowledge  Test  Safety:  First  aid.  electrical  safety.  CPR  (Safety) 

Knowledge  Test  Basics:  Ohm’s  law.  power  schematics,  fluke  77.  AN  circuit  construction.  DC  series  circuits  (Basics) 
Knowledge  Test  DC  parallel  circuit  construction  &  \ariational  analysis,  serics/parallel  characteristics.  DC  troubleshooting  (DC 
Circuit) 

Knowledge  Test  AC  generation,  v.'aveform  anal v’sis /oscilloscope,  electromagnetism,  miscellaneous  test  ec^uipment  (AC  Circuit  ) 
Knowledge  Test  inductance  fundamcntals/reactancc.  RL  circuits,  transformers  (Inductance) 

Knowledge  Test  Capacitance  fundamcntals/reactancc.  RC  circuits.  RLC  circuits  (Capacitance) 

Comprehensive  Knowledge  Test  L  Content  from  tests  002  to  008  fComp  1) 


2 


OlU 

on 

012 

013 

014 

015 

016 


Knowledge  Test  Engineering  Admin:  Tag-out.  preventative  maim.,  personnel  qualification  sysi./tools/fastcncrs,  cables 
(Engincenng) 

Knowledge  Test  Generators:  Steam  c>'cle.  AC  &  DC  generators,  generator  maintcnance/iechnical  manuals  (Generators) 
Knowledge  Test  Distribution:  Introduction  to  distribution.  EOSS.  swiichboards/shore  power  (Distribution) 

Knowledge  Test  Motors:  DC  &  AC  motors,  motor  maintenance  (Motors) 

Knowledge  Test  Controllers:  Introduction  to  controllers.  AC  controllers  (Controllers) 

ComprcbcDave  Knowledge  Test  2:  Content  from  tests  010  to  014  (Comp  2) 

Knowledge  Test  AC/R  &  Galley:  Refrigeration  &  air  conditioning,  galley  auxiliary  (Galley) 


On  March  6.  1991.  the  school  changed  the  sequence  of  the  Lighting,  Small  Craft,  and  Auxiliary  tests.  Form  1  covers  students  entering 
the  school  before  this  date.  Form  2  addresses  students  entering  the  school  after  March  6,  1991. 


017  Knowledge  Test 

Form  1:  Lighting  distribution  navigational  lighting  (Lighting) 

Form  Smallcraft:  Batteries,  starting/charging  system  (Small  Craft) 

018  Knowledge  Test 

Form  1:  Smallcraft:  Batteries,  staning/charging  system  (Small  Craft) 

Form  2-  Miscellaneous  Aualiary-  Degaussing,  cathodic  protection,  vaK'c  operators  (Auxiliiary) 

019  Knowledge  Test 

Form  1:  Miscellaneous  Auxiliary:  Degaussing,  cathodic  protection,  valve  operators  (Auxiliiary) 
Form  2  Lighting  distribution  &  navigational  lighting  (Lighting) 

020  CocnpicbcQsivc  Knowledge  Test  2  Content  from  tests  016  to  019 


Phase  3 


021 

022 

023 

024 


Knowledge  Test  Electronics:  P.N.  junction  theory,  special  devices,  transistor  theory  (Elect  1) 

Knowledge  Test  Electronics:  Power  supplies,  transformers  &  reaifiers,  AMP  operation/transistor  configuration,  integrated 
circuits,  introduction  to  logics  (Elect  2) 

Knowledge  Test  Elevators:  Introduction  to  basic  components,  power  supply  &  signal  converters,  operations  of  gale  circuits  &, 
set-reset/pulser/stepper/inpui  dcvices/magnetic  controller,  flow  charts  &  elevator  operations  (Elevators) 
Compicliettsive  Knowledge  Test  4:  Content  from  test  #(X)2  to  #023  (Comp  4) 


Results  and  Discussion.  We  began  by  testing  for  an  order  effect  due  to  the 
change  in  test  order  of  the  Lighting,  Small  Craft  and  Auxilliary  exams.  We 
separated  the  students  into  Form  1  and  Form  2,  based  upon  whether  they  entered 
the  school  before  or  after  the  test  order  change.  We  then  conducted  a 
multivariate  analysis  of  variance  (MANOVA)  to  test  the  correlation  matrices  of 
the  test  scores.  A  test  for  homogeneity  of  covariance  matrices  revealed  that  the 
two  forms  had  significantly  different  covariance  matrices  (F=1.67,  fi  <  .001). 
The  test  of  means  also  differed  significantly  across  the  two  forms  (F*6.97,  £  < 
.001).  These  results  suggest  that  the  two  samples  come  from  two  distinct 
populations.  Univariate  F-tests  revealed  nine  of  the  tests  to  differ  between 
forms  at  the  £  <  .05  level  (Math  Pretest,  Math,  DC  Circuit,  AC  Circuit, 
Capacitance,  Distribution,  Galley,  Lighting,  and  Auxill iary) .  In  addition  to  two 
of  the  re-ordered  tests  (Small  Craft  and  Lighting),  these  include  seven  exams 
occurring  earlier  in  the  curriculum.  This  finding  suggests  that,  in  addition  to 
the  test  re-ordering,  the  two  groups  also  differ  in  some  other  manner. 

To  test  whether  ability  accounted  for  this  difference,  RGI  conducted  a 
multivariate  analysis  of  covariance  (MANCOVA)  using  the  student’s  pre-enlistment 
ASVAB  scores  as  covariates.  After  parti  ailing  out  the  variance  due  to  ASVAB 
scores,  a  test  for  homogeneity  of  covariance  matrices  revealed  different  matrices 
for  the  two  forms  (F=1.37,  £  <  .001).  Test  means  also  differed  across  the  two 
forms  after  controlling  for  ASVAB  scores  (F=7.33,  £  <  .001).  These  results 
suggest  that  the  difference  between  the  two  groups  is  due  to  something  other  than 
differences  in  ability.  Based  on  these  findings,  RGI  analyzed  each  group 
separately. 

To  further  evaluate  the  conclusion  of  differences  between  groups,  we 
computed  effect  sizes  (ES)  as  measures  of  the  practical  significance  of  the 
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Tabic  35 


Descnpuve  Statistics  for  EM  Variables 


Test 

n 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Form  1 

Math  Pretest 

748 

57.444 

16.137 

.057 

-  .262 

4.300 

100,00 

Math 

759 

93.361 

7.801 

-2.409 

8.372 

4330 

100.00 

Safct\' 

771 

90.462 

7.636 

-1.013 

.853 

59.40 

100,00 

Basics 

767 

87336 

9.433 

-1.279 

2.273 

39.60 

100.00 

DC  Circuit 

1€A 

80.609 

13.236 

-.853 

.421 

3230 

100.00 

AC  Circuit 

762 

87.087 

10.219 

-1.439 

4.356 

1230 

100.00 

Inductance 

762 

87.828 

9.956 

-1.W2 

1.110 

40.00 

100.00 

Capacitance 

762 

87.025 

11338 

-1.386 

2.621 

25.00 

100.00 

Comp  1 

762 

86,201 

8.865 

-.675 

.091 

55.80 

100.00 

Engincenne 

759 

85.368 

10.227 

-1.070 

2.053 

26.30 

100.00 

Generators 

758 

87.421 

9.172 

-1.381 

3.921 

27.10 

100.00 

Distribution 

755 

88.266 

8.354 

-1.097 

1.572 

50.00 

100.00 

Motors 

753 

85.488 

10.681 

-1.140 

2.386 

30.00 

100.00 

Controllers 

745 

86.703 

9.664 

-1.222 

2.544 

29.20 

100.00 

Comp  2 

744 

87.059 

7.772 

-  .875 

1.322 

50.00 

100.00 

Galley 

743 

85.869 

10391 

-1.138 

2.120 

27.30 

100.00 

Lighting 

741 

87.632 

10.479 

-1.237 

1.708 

35.00 

100,00 

Small  Craft 

740 

88.487 

8.889 

-  .888 

.555 

52.80 

100,00 

Auxilliary 

739 

87.761 

9.158 

-1.161 

2.351 

33.30 

100.00 

Comp  3 

739 

84.209 

10.115 

-  .730 

.607 

42,30 

100.00 

Elect  1 

736 

89,141 

9.079 

-1.356 

3303 

2730 

100.00 

Elect  2 

734 

89.444 

9.262 

-1.669 

4307 

33.90 

100.00 

Elevators 

731 

87.758 

10.754 

-1.905 

5i08 

22.70 

100.00 

Comp  4 

731 

83.432 

8.232 

-  366 

.229 

53.00 

100.00 

Form  2 

Math  Pretest 

182 

65.930 

13.620 

.054 

-  .017 

24.60 

98.60 

Math 

185 

94.659 

6.762 

-2.448 

8.141 

5630 

100.00 

Safety 

187 

89.736 

8.550 

-2.389 

11.794 

31.30 

100.00 

Basics 

186 

88.963 

8.781 

-  ,846 

.106 

60.40 

100,00 

DC  Circuit 

186 

83.311 

11.813 

•  .795 

.853 

33.30 

100.00 

AC  Circuit 

184 

89.769 

7.783 

-  .703 

.026 

63.00 

100.00 

Inductance 

184 

87.989 

9.356 

-.732 

,033 

55.00 

100.00 

Capacitance 

184 

89382 

9399 

-1.054 

.682 

60.40 

100.00 

Comp  1 

184 

86353 

8.711 

-.647 

-  .349 

65.40 

100.00 

Engineering 

183 

84.393 

9.136 

-  384 

.176 

57.90 

100.00 

Generators 

182 

88.871 

8.470 

-1.047 

1.410 

56.30 

100.00 

Distribution 

181 

90.244 

7.183 

-1.107 

1.965 

58.30 

100.00 

Motors 

181 

87.155 

8.936 

-.738 

-  .018 

60.00 

100.00  • 

Controllers 

181 

87.945 

8.437 

-  .872 

1.042 

52.10 

100.00 

Comp  2 

181 

87321 

6.640 

-.476 

-.068 

63.90 

100.00 

Galley 

181 

83.664 

11.488 

-  .684 

.131 

50.00 

100,00 

Sm  Craft 

181 

84.309 

10.352 

-.709 

.102 

50.00 

100.00 

Auxilliary 

181 

87.039 

9.403 

-  .847 

.247 

5830 

100.00 

Lighting 

181 

83360 

10.796 

-.775 

.406 

45.80 

100.00 

Comp  3 

181 

82.921 

10.416 

-.410 

-  .424 

57.70 

100.00 

Elect  1 

178 

88.989 

9.881 

-2.105 

9.391 

25.00 

100.00 

Elect  2 

178 

88.842 

9.229 

-2.330 

9.131 

33.90 

100.00 

Elevators 

178 

88.278 

8.405 

-.908 

.631 

59.10 

100.00 

Comp  4 

177 

83.415 

8.048 

-.451 

.213 

54.00 

99.00 

60 


differences  between  the  unadjusted  means  across  group.  Cohen  and  Cohen  (1977) 
state  that  .2  represents  a  small  effect  size,  .5  is  medium,  and  .8  is  large.  For 
a  total  of  24  comparisons  between  the  EM  groups,  we  found  16  ESs  within  the  .00 
to  .19  range,  6  ESs  within  the  .20  to  .49  range,  and  2  ESs  within  the  .50  to  .79 
range.  These  results  show  that  91%  of  the  ESs  fall  below  a  value  of  .50.  Thus, 
while  the  MANOVA  tests  indicate  that  the  groups  differ  from  one  another  in  terms 
of  statistical  significance,  the  ESs  suggest  small  magnitudes  of  the  differences, 
in  terms  of  practical  significance. 

Within  the  EM  curriculum.  Form  1  contains  students  who  took  the  course 
before  the  test  order  changes  of  March  6,  1991  (n=774).  Form  2  contains  students 
taking  the  EM  course  after  the  test  re-ordering  of  March  6,  1991  (n=187).  Form 
1  included  58  cases  with  missing  data.  RGI  excluded  these  cases  from  some  of  the 
analyses,  resulting  in  a  final  sample  of  716  cases  for  Form  1.  Form  2  included 
12  cases  with  missing  data.  Likewise,  we  excluded  these  cases  from  some  of  the 
analyses,  resulting  in  a  final  sample  size  of  175  cases  for  Form  2. 

RGI  computed  descriptive  statistics  for  each  of  the  tests.  Table  35 
presents  these  statistics,  listing  them  separately  for  Form  1  and  Form  2.  Next, 
we  conducted  a  factor  analysis  on  each  form  separately.  We  included  the  set  of 
all  tests  except  Math  Pretest  in  these  analyses.  School  instructors  indicated 
they  gave  this  test  prior  to  any  instruction.  Resulting  scree  tests  and  factor 
loadings  for  the  two  forms  were  similar,  both  indicating  a  two-factor  solution. 

Table  36  shows  the  commonalities,  factor  loadings,  and  factor  score 
coefficients  for  Form  1  and  Form  2.  For  both  forms,  Phase  2  and  Phase  3  tests 
loaded  highly  on  the  primary  factor  and  Phase  1  tests  loaded  highly  on  the  second 
factor.  RGI  labeled  the  factors  "Technical"  and  "Basics"  (respectively)  to 
reflect  this  finding. 
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Table  36 


■Communahties.  Factor  Loading;  and  Factor  Score  Coefficients  for  EM  Variables 


Factor  Loadings 

Factor  Score  Coefficients 

Communalitv' 

Form  1 

Form  2 

Form  1 

Form  2 

Test 

Form  1  Form  2 

Fad*  Fac2'’ 
Technical  Basics 

Fad  Fac2 

Technical  Basics 

Fad  Fac2 

Technical  Basics 

Fad  Fac2 
Technical  Basics 

Math 
Safety 
Basics 
DC  Circuit 
AC  Circuit 
Inductance 
Capacitance 
Comp  1 

Engincenng 

Generators 

Distnbution 

Motors 

Controllers 

Comp  2 

Galley 
Lighting 
Small  Craft 
Auxilliary 
Comp  3 

Elect  1 
Elect  2 
Elevators 
Comp  4 


.212  .077 

.155  .297 

.486  .471 

.478  .415 

.407  .434 

.425  .501 

.360  .605 

.437  ^50 

.273  .330 

.408  .450 

.385  .314 

.329  .361 

.366  .405 

.444  .437 

.440  .600 

.445  .402 

.471  .370 

.424  .413 

.365  .357 

.303  .271 

.321  .123 

.237  .192 

.403  .351 


.213 

.408 

.146 

.235 

.248 

.306 

.448 

.309 

.240 

.655 

.365 

381 

.222 

.655 

.389 

314 

.234 

.693 

.232 

.617 

.229 

.610 

.357 

.688 

.179 

.572 

.176 

.758 

.201 

.630 

.245 

.700 

.475 

.217 

ilO 

.264 

.506 

.390 

.571 

.351 

.444 

.433 

.457 

.325 

.470 

.328 

.484 

.359 

.462 

.390 

.575 

.273 

.523 

.412 

343 

.378 

.611 

.257 

.769 

.096 

.641 

.187 

371 

.275 

.657 

.200 

324 

.309 

.635 

.145 

331 

.362 

.578 

.178 

.451 

.391 

.429 

.^5 

.305 

.421 

.408 

.252 

.169 

.307 

.451 

.184 

.399 

.181 

.489 

.405 

352 

.214 

-.014 

.068 

-.000 

.022 

.016 

.032 

.049 

,011 

-.080 

.225 

-.036 

.147 

-.084 

.225 

.026 

.089 

-.058 

.164 

-.045 

.157 

-.067 

.184 

-.079 

.249 

-.068 

.161 

-.169 

.338 

-.085 

.207 

-.066 

.207 

,086 

-.015 

.085 

-.009 

.083 

.028 

,113 

.015 

.047 

.059 

.068 

.003 

.072 

.100 

.057 

.004 

.058 

.040 

.129 

-.043 

,087 

.036 

.108 

.005 

.150 

-.042 

,367 

-.223 

.180 

-.081 

-119 

-.037 

.194 

-.081 

.073 

-.002 

.186 

-.093 

.113 

-.007 

,136 

-.060 

.061 

.017 

.067 

.019 

.008 

.056 

.102 

-.018 

.002 

.036 

.084 

-.022 

.066 

r.009 

.077 

.035 

.126 

-.032 

Note.  For  Form  Lji  =  716.  For  Form  2,  jn 
^  Fad  =  Factor  1. 

^  Fac2  =  Factor  2. 


While  the  factors  appear  to  have  the  same  underlying  factors,  we  computed 
congruence  coefficients  which  indicate  that  the  factor  patterns  are  different 
(Horsuch,  1983,  p.  285,  Congruence  =  .21  p<.05  for  Factorl,  and  Congruence  =  .23, 
p<.05  for  Factor2). 

We  sought  to  develop  composite  variables  which  would  capture  essential 
elements  of  the  criterion  variance  and  also  represent  meaningful  dimensions 
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Table  3* 


Deserptive  Statistics  for  EM  Comrosite  V*anahles 


Composite 

n 

Rel* 

Mean 

Std  Dev 

Skew  Kurtosis  Min 

Max 

Form  1 


Sum 

730 

.89 

86.851 

6.077 

.  .331 

-  .359 

68.60 

99.94 

Sum  2" 

716 

.89 

87.188 

5.786 

-  .245 

-  .289 

67.62 

99.63 

FSG 

773 

S/A 

87.805 

4.884 

.078 

-  .331 

67.10 

99.70 

716 

.92 

86.974 

5i85 

-.150 

-  500 

69.17 

99.76 

Basics® 

74S 

.82 

87.383 

6.667 

-  .576 

-  .204 

59.96 

100.00 

Technical" 

730 

.89 

87.035 

6.000 

-  .291 

-  .433 

70.23 

99.93 

Form  2 

Sum  1 

177 

.91 

86.601 

5.813 

-  .060 

-  ,583 

70.89 

98.39 

Sum  2 

178 

.91 

87.716 

5.747 

.  .139 

-  .766 

73.46 

98.94 

FSG 

187 

N/A 

87.689 

5.006 

-  .131 

.482 

67.00 

98.40 

FSG2 

175 

.92 

87.049 

5.369 

.014 

•  .721 

73.33 

9859 

Basics 

182 

.85 

88.835 

6.089 

-  .368 

-.675 

71.97 

99.47 

Technical 

177 

.87 

86.449 

5.658 

-  .003 

-  .643 

73.95 

98.90 

^  These  are  composites  based  on  coonunality  estimates  and  the  procedures  described  in  the  text. 

^Suml;  (Average  of  all  tests  with  Factor  I  loadings  greater  than  or  equal  to  .30). 

^  Suin2:  (Average  of  all  tests  with  Factor  2  loadings  greater  than  or  equal  to  .30). 

^FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

®  Basics:  Average  score  of  Phase  1  tests:  (Math,  Safety,  Basics,  DC  Circuit.  AC  Circuit.  Inductance, 
Capacitance,  Comp  1).  Does  not  include  Math  Pretest. 

^Technical:  Average  score  of  Phase  2  &  3  tests:  (Engineering,  Generators,  Distribution,  Motors,  Controllers, 
Comp  2,  Galley.  Lighting,  Small  Craft.  Auxilliary,  Comp  3,  Elect  1,  Elect  2,  Elevators,  Comp  4). 


useful  to  the  EM  school.  Correlations  with  the  empirical  factors  would 
illustrate  the  degree  to  which  the  simpler  unit-weighted  composites  could 
represent  the  empirically  defined  factor.  Accordingly,  we  developed  a  Technical 
composite  (the  average  of  all  Phase  2  and  Phase  3  tests)  to  represent  Factor  1, 
and  a  Basics  composite  (the  average  of  all  Phase  1  tests  except  Math  Pretest)  to 
represent  Factor  2.  We  also  created  Sum  1  and  Sum  2  composite  variables  from  an 
average  of  all  tests  loading  .3  or  greater  on  either  Factor  1  or  Factor  2, 
respectively.  Since  the  EM  school  computed  the  FSG  using  75  as  the  minimum 
score,  RGI  calculated  its  own  final  grade  (FSG2)  using  initial  test  scores  and 
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Tabic  3S 


Correlations  Among  EM  Comtx>S!tes  and  Factors 


Composite 

Faciorl 

Facior2 

Suml* 

Sum2^ 

FSG 

FSG2' 

Basics^ 

Technical® 

Factor  1 

, 

.16 

.87 

.68 

.79 

.79 

oo 

.90 

Factor  2 

.17 

— 

.62 

.81 

.70 

.72 

.90 

.54 

Sum  1 

.90 

-52 

— 

.95 

.98 

.99 

.83 

.98 

Sum  2 

.60 

.88 

.85 

— 

.96 

.97 

.92 

.91 

FSG 

.79 

.71 

.93 

.94 

.99 

.87 

.96 

FSG2 

.79 

.73 

.94 

.95 

.98 

.89 

.96 

Basics 

.36 

.95 

.64 

.92 

.82 

.84 

... 

.74 

Technical 

.93 

.51 

.99 

.84 

.94 

.95 

.64 

Sot^  Coefficients  below  the  diagonal  correspond  to  Form  1  (n  »  716).  Coefficients  above  the  diagonal 
correspond  to  Form  2  (n  *  175).  ” 

^Suml:  (Average  of  all  tests  with  Factor  1  loadings  greater  than  or  equal  to  .30). 

^Sura2:  (Average  of  all  tests  with  Factor  2  loadings  greater  than  or  equal  to  .30). 

*SG2:  Finiai  School  Grade  computed  using  initial  test  scores. 


basics :  Average  score  of  Phase  1  tests;  (Math,  Safety.  Basics,  DC  Circuit.  AC  Circuit,  Inductance, 

Capacitance.  Comp  1). 

Technical:  Average  score  of  Phase  2  &  3  tests:  (Engineering,  Generators,  Distribution,  Motors,  Controllers, 
Comp  2,  Galley.  Lighting,  Small  Craft,  Auxilliary,  Comp  3,  Elect  1.  Elect  2,  Elevators,  Comp  <» ) . 


listed  both  as  composite  variables.  We  employed  the  factor  scores  (Factor  1  and 
Factor  2)  as  additional  composites.  Table  37  shows  the  descriptive  statistics 
for  each  of  the  composite  variables  noted  above.  The  sample  size  varied  for  the 
composites  depending  on  the  number  of  tests  missing  for  each  case. 

Next,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  38  presents  these  correlations.  Notice  that  FSG  and  FS62 
correlated  .98  and  .99  for  Form  1  and  Form  2,  respectively.  This  offers  no 
support  for  using  initial  test  scores  in  FSG.  For  both  Form  1  and  2,  the 
Technical  composite  showed  the  largest  correlation  with  Factor  1  (.93  and  .90, 
respectively),  and  the  Basics  composite  showed  the  largest  correlation  with 
Factor  2  (.95  and  .90,  respectively).  The  Suml  and  Sum2  variables  did  not 
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correlate  as  high  with  their  associated  factor  scores.  Further,  Suml  and  Sum2 
had  higher  correlations  with  opposite  factor  scores  than  did  the  Basics  and 
Technical  composites.  Finally,  for  Forms  1  and  2,  respectively,  FS6  correlated 
.82  and  .87  with  Basics  and  .94  and  .96  with  Technical. 

Recommendations.  In  addition  to  FSG,  use  an  average  of  Phase  1  tests. 
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Electronic  Technician  Phase  1  (ETl) 

Instruction  for  the  Electronic  Technician  School  takes  place  over  the 
course  of  two  phases.  Students  first  attend  Phase  1,  located  in  Orlando, 
Florida.  Upon  satisfactory  completion  of  Phase  1,  students  move  on  to  Phase  2, 
located  in  Great  Lakes,  Michigan.  The  present  section  covers  the  Phase  1 
curriculum,  which  we  labeled  ETl.  The  subsequent  section  addresses  the  Phase  2 
curriculum  (labeled  ET2). 

Description  of  Variables.  The  ET  school  administers  the  following  types 
of  tests  to  assess  student  progress:  lab  quizes,  blitzes,  homework,  knowledge 
exams,  performance  exams,  and  cumulative  exams.  Because  so  many  of  the  cases  had 
incomplete  data  on  these  tests,  we  restricted  our  analyses  to  the  knowledge  and 
performance  exams.  These  were  also  the  only  tests  included  in  the  FSG.  Valid 
ranges  for  the  test  scores  vary  (the  maximum  possible  score  ranges  from  5  to 
100).  Table  39  presents  these  tests  and  their  corresponding  descriptions. 

The  school  requires  a  minimum  passing  grade  of  63%  on  all  quizzes  and  exams 
for  graduation.  Students  who  score  below  63%  receive  remediation  and  a  retest. 
The  school  allows  one  retest  and  assigns  students  who  fail  the  retest  to  an 
Academic  Review  Board.  Students  receive  a  score  of  63%  when  they  pass  a  retest 
or  repeat  an  area  of  instruction.  However,  in  the  present  study,  RGI  entered  all 
initial  test  scores  into  the  database. 

Sample.  The  initial  sample  consisted  of  130  students  who  began  training 
between  April  and  October,  1990.  We  exluded  students  who  did  not  take  every  test 
from  some  of  the  analyses.  Consequently,  our  final  sample  contained  66  cases. 

Results  and  Discussion.  Because  the  valid  range  differed  among  the  tests, 
we  converted  each  test  to  a  scale  of  0-100  by  transforming  them  into  percent 
correct  scores.  Then  we  computed  descriptive  statistics  for  each  of  the  test 


Tabic  yi 


Descnrtions  of  FTl  Variahles 


Tcsi 


Description  { Name  -  T>*pc  of  Test)* 


10401-10409 

:9401-:9409 
20401-20409 
39401-39409 
30401-30409 
49401-49409 
40401-40409 
50  •1-504C'9 
60401-60409 

70401-70409 

80401-80409 

90401-90409 


Koowledge:  Learning  d:  study  skills,  math  for  basic  electronics/scienttfic  calculator/graphs/maps,  electrical  safety  &, 
firefighting  procedures,  magnetic  &  chemical  properties  of  matter  (Basics-K). 

Perfonnaooe:  Electrical  cnerg>'.  series  &  parallel  DC  circuits  (DC  Circ-P). 

Koowtedge:  Electrical  energy,  series  &  parallel  DC  circuits  (DC  Circ-K). 

Performance;  AC  theory,  test  equipment,  capacitance,  inductance,  transformers  (AC  Theory-P). 

Knowledge:  AC  thcorv%  test  equipment,  capacitance,  inductance,  transformers  (AC  Theor>'-K). 

Perfonnanoe:  RLC  circuit  analysis,  tuned  circuits  &  filters,  integrators  &  deffcrennaiors.  semiconductor  diodes, 
limiters  i  AC  Circ-P). 

Knowledge:  RLC  circuit,  tuned  circuii/filter.  imcgrator/differentiator.  semiconductor  diode,  limiters  (AC  Circ  K). 

Knowledge:  Clampers,  transistors,  audio  frequency  amplifiers,  special  amplifiers  (Trans-KV 

Knowledge:  Electron  tube  diodes  &.  triodes.  solid  state  power  supplies,  special  semiconductor  devices  (Power 
Sup-K). 

Knowledge:  Wave  generating  circuits:  Multivibrators,  sawtooth  generators,  oscillators  (Wave  Circ-K). 

Knowledge:  Decibels.  AM  receivers,  radio/intermediate  frequency  amplifiers  &  mixers,  receiver  alignment  (AM 
Reevr-K). 


Knc^idedge:  AM  transmitters,  radio  frequency  power  amplifiers,  amplitude  modulation,  transceivers,  transmission  lines 
and  antennas,  transmission  lines,  radio  frequency  radiation  hazards  (Xmtr-K). 


^  K:  Knowledge,  P:  Ferfortnance . 


variables.  Table  40  presents  these  statistics.  Because  many  of  the  students 
missed  several  of  the  exams,  we  were  not  able  to  conduct  a  factor  analysis  for 
this  school.  Also,  because  of  the  small  sample  sizes  for  the  AC  Theory  and  Power 
Sup  tests  (for  each  test,  n  *  21),  we  did  not  include  them  in  further  analyses. 

Next,  we  sought  to  develop  composite  variables  which  would  capture 
essential  elements  of  the  criterion  variance  and  also  represent  meaningful 
content  dimensions  useful  to  the  ET  school.  Table  41  presents  Pearson 
correlation  coefficients  for  the  test  variables  and  FS6.  These  correlations  show 
the  pattern  of  relationships  between  the  individual  tests. 
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I'aMc  4'- 


Descriptive  Statistics  for  ETl  Variables 


Variable 

H 

Mean 

Std  Dev 

Skew 

Kunosis 

Min 

Max 

Basics-K 

130 

90.092 

6.833 

«1.148 

1.075 

66.00 

100.00 

DC  Circ-P 

127 

93.602 

7.814 

-1.465 

2.266 

6230 

100.00 

DC  Circ-K 

128 

S3.2^ 

11.089 

-2,619 

15362 

08.00 

100.00 

AC  Thcory-P 

129 

85.969 

14.335 

-1.475 

3.011 

30.00 

AC  Thcorv'-K 

021 

79.429 

09332 

-  .254 

-  .117 

58.00 

096.00 

AC  Circ-P 

129 

84.961 

20.771 

-1.408 

1.261 

20.00 

100.00 

AC  Circ-K 

129 

79319 

12.494 

-1.291 

2.381 

26.00 

098.00 

Trans-K 

128 

74.672 

11368 

-.795 

1.820 

26.00 

096.00 

Power  Sup-K 

021 

77.714 

10.813 

-  .100 

-  .545 

56.00 

096.00 

Wave  Circ-K 

128 

70.828 

14.232 

-.613 

.342 

26,00 

094.00 

AM  Rccvt-K 

128 

"6.109 

10378 

-.320 

-  314 

48.00 

096,00 

Xmir-K 

067 

"8.507 

09.486 

-  .499 

•  .123 

54.00 

098,00 

Table  41 

Correlations  Among  ETl  FSG  and  Test  V^ariables 


Variable 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1.  FSG 

2.  Basics-K 

.33 

3.  DC  Circ-P 

.28 

.21 

4.  DC  Circ-K 

.71 

.37 

.19 

5.  AC  T7ieorv-P 

.46 

.03 

.15 

.32 

6.  AC  Circ-P 

.21 

-.12 

-.08 

.05 

.09 

7.  AC  Circ-K 

.69 

,22 

.19 

35 

.34 

.16 

8,  Trans-K 

.79 

.27 

.17 

.60 

.28 

.18 

.45 

9,  Wave  Circ-K 

.66 

.27 

.13 

,33 

.21 

.13 

.42 

.47 

10.  AM  Reevr-K 

.68 

-.05 

.13 

.38 

.15 

.10 

.30 

38 

.39 

11.  Xmtr-K 

.73 

.19 

.18 

.39 

.28 

-.04 

.47 

39 

33 

37 

Note,  n  «  66. 
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Descnptive  Statistics  for  ETl  Composite  V'ariables 


Composite 

n  Rel® 

Mean 

Sid  Dev 

Skew 

Kurtosis 

Min 

Max 

FSG 

130 

81.212 

6.174 

.189 

-.637 

67J0 

095.40 

FSG2‘’ 

066 

79.927 

7.052 

-.247 

00 

61.70 

095.10 

CTest' 

067 

88.130 

6.972 

-.423 

-  .166 

62.29 

095.14 

PTesi° 

127 

88.130 

9.278 

-.593 

-  .521 

65.83 

100.00 

Phase  1® 

126 

88.262 

9.278 

-.745 

.399 

66.19 

99  JO 

Phase  2^ 

128 

79.870 

10.554 

-.603 

-  .265 

50.67 

97.33 

Phase  3® 

067 

76.925 

9.126 

-.421 

-.052 

54.00 

94.67 

^  These  are  coxposites  based  on  conxDunality  estimates  determined  through  regression  procedures. 
“  FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

*  KTests:  .V/erage  of  all  knowledge  test  scores. 

^  rlests :  Average  of  all  quiz  scores. 

®  Phase  i:  .V/erage  of  Basics-K,  DC  Circ“P,  DC  Circ“K.  and  AC  Theory-P  test  scores. 

•  Phase  Z:  Average  of  AC  Circ-P,  AC  Circ-K,  and  Trans-K  test  scores. 

®  Phase  3:  Average  of  Wave  Circ-K,  AM  Recvr-K,  and  Xxntr-K  test  scores. 


In  order  to  examine  whether  test  method  could  explain  aspects  of  school 
performance,  we  developed  a  knowledge  test  (KTest)  composite  and  a  performance 
test  (PTest)  composite.  Since  the  ETl  school  computes  the  FS6  using  63  as  the 
minimum  score,  RGI  calculated  its  own  final  grade  (See  Appendix  F  for 
calculation)  using  initial  test  scores  and  listed  both  as  composite  variables. 
Finally,  we  developed  three  more  composites  based  on  averages  of  the  test 
material  included  in  the  school’s  cumulative  exams:  Phase  1  (Basics-K,  DC  Circ- 
P,  DC  Circ-K,  and  AC  Theory-P),  Phase  2  (AC  Circ-P,  AC  Circ-K,  and  Trans-K),  and 
Phase  3  (Wave  Circ-K,  AM  Recvr-K,  and  Xmtr-K) .  Table  42  presents  the  descriptive 
statistics  for  each  of  the  composites  mentioned  above.  For  comparison,  we  placed 
all  the  composite  variables  on  a  scale  of  0-100  by  transforming  them  into  percent 
correct  scores.  The  sample  size  varied  for  the  composites  depending  on  the 
number  of  tests  missing  in  each  case. 
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Correlations  Among  ETl  Composites 


S'ariable 

FSG 

FSG2 

KTests 

PTests 

Phase  1 

Phase  2 

1.  FSG 

3.  fsg:® 

.95 

3.  K  Tests- 

.95 

.99 

4.  PTcsts‘= 

.44 

.38 

.30 

5.  Phase 

.72 

.61 

.65 

.47 

6.  Phase  2® 

.62 

.60 

S6 

.83 

.34 

7.  Phase  3- 

.85 

.95 

.90 

.23 

.43 

.43 

Note .  n  *  66 . 

^  FSG2:  See  Appendix  _  for  computation. 

^  KTests:  Average  of  all  knowledge  test  scores. 

PTests  ;  Average  of  all  perfor.mance  scores. 

Phase  i:  Average  of  Basics-K.  2C  Circ-P.  DC  Circ-K.  and  AC  Theory-P  test  scores. 
®  Phase  2:  Average  of  AC  Circ-r.  AC  Circ-K.  and  Trans-K  test  scores, 
base  3:  Average  of  Wave  Circ-K.  AM  Recvr-K,  and  Xmtr-K  test  scores. 


Then,  we  computed  Pearson  correlation  coefficients  for  the  composites.' 
Table  43  presents  the  composite  variable  correlations.  Since  FS6  and  FSG2 
correlate  .95,  this  offers  little  support  for  using  initial  test  scores  to 
compute  FSG. 

In  reviewing  the  results  with  the  phase  variables,  we  find  that  Phase  1 
correlates  moderately  (.47  to  .72)  with  FSG  and  the  knowledge  and  performance 
composites.  Phase  2  correlates  highest  (.83)  with  the  performance  composite,  and 
Phase  3  correlates  very  highly  (.85  and  .90,  respectively)  with  FSG  and  the 
knowledge  composite. 

Finally,  FSG  correlates  .95  with  KTests  and  .44  with  PTests.  These 
findings  support  using  FSG  as  an  indicator  of  for  knowledge  tests  but  suggest 
using  a  separate  indicator  for  performance  tests. 

Recommendations.  In  addition  to  FSG,  use  PTests  (an  average  of  DC  Circ-P, 
AC  Theory-P,  and  AC  Circ-P). 
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Electronic  Technician  Phase  2  (ET2) 

Instruction  for  the  Electronic  Technician  School  takes  place  over  the 
course  of  two  phases.  Students  first  attend  Phase  1,  located  in  Orlando, 
Florida.  Upon  satisfactory  completion  of  Phase  1,  students  move  on  to  Phase  2, 
located  in  Great  Lakes,  Michigan.  The  preceeding  section  (labeled  ETl)  covers 
the  Phase  1  curriculum.  This  section,  ET2,  addresses  the  Phase  2  curriculum. 

Description  of  Variables.  The  ET  school  administers  the  following  types 
of  tests  to  assess  student  progress:  blitzes,  homework,  knowledge  exams,  and 
performance  exams.  Because  so  many  of  the  cases  had  incomplete  data  on  these 
tests,  we  restricted  our  analyses  to  the  knowledge  and  performance  exams.  These 
were  also  the  only  tests  included  in  the  FSG.  All  tests  have  a  valid  range  of 
0-100.  Table  44  presents  these  tests  and  their  corresponding  descriptions. 

The  school  divided  the  course  into  training  areas,  also  shown  in  Table  44. 
On  March  31,  1991,  the  school  incorporated  a  new  area  (AC  Primary  Power 
Distribution  and  Supplies)  into  its  curriculum.  Students  entering  the  ET2  course 
after  the  addition  of  the  new  area  received  less  time  per  training  area. 

The  school  requires  a  minimum  passing  grade  of  63%  on  all  quizzes  and  exams 
for  graduation.  Students  who  score  below  63%  receive  remediation  and  a  retest. 
The  school  allows  one  retest  and  assigns  students  who  fail  the  retest  to  an 
Academic  Review  Board.  Students  receive  a  score  of  63%  when  they  pass  a  retest 
or  repeat  an  area  of  instruction.  However,  in  the  present  study,  RGI  entered  all 
initial  test  scores  into  the  database. 

Sample.  The  entire  sample  consisted  of  114  students  who  began  training 
between  October  1990  and  December  1991.  We  excluded  students  who  did  not  take 
every  test  from  some  of  the  analyses.  Consequently,  our  final  sample  contained 
83  cases . 
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Table  44 


Desenpnons  of  FT2  Variables 


Area  Test  Dcscripuon  (Name  -  Tvpe  of  Test)* 


3 

n 

11 

12 

12 


501 

502 
602 

503 
603 

511 
Ml 

512 
612 


Knowledge:  FM  fundamentals  and  DC  circuit  analysis,  narrow  band  frequency  modulator,  receiver  functional 
operation,  biased  semiconductor  diode  PN  junction,  basic  amplifier  (P4/DC-K). 

Knowledge:  Single  sideband  receiver  (SSB)  functional  operations  and  receiver  fundamentals,  receiver  front  end 

circuits,  intermediate  frequency  amplifiers,  demodulator,  audio  amps/ reproduct  ion  devices  (SSB  Reevr-K). 

Performance:  Single  sideband  receiver  troubleshooting  (SSB  Reevr-P). 

Knowledge:  SSB  transmit  functional  circuits,  oscillators.  SSB  modulation,  first  intermediate  frequency  amps,  four 
diodc/high  frequency  balanced  mixer  &  RF  amplifier  (SSB  Xmir-K). 

Performance:  Single  sideband  transmitter  troubleshooting  (SSB  Xmtr-P). 

Knowledge:  R-1051B/URR  high  frequency  receiver  functional  block  diagrams,  AC/DC  distribution  & 

antenna/amplifier/transistor/synthcsizcr/rcccrver  mode  selector.  EF/AF  amplifier  (AN/WRC-K). 

Performance:  ASP^’KC  -  IB  receiver,  troubleshooting  pnnciples  for  RM051B/URR  (AN/WRC-P). 

Knowledge:  AN/ljRT-24  operations,  high  frequency  transmitter  T-827  chassis  &  main  frame  circuit  analysis.  FSK 
tone  generator,  transmit  mode  selector/IF,  AF  RF  amp,  AM-3007/URT.  CU-937/UR  (AN/URT-K). 

Performance:  A.N’/URT  -  24  operations  and  troubleshooting  (AN/URT-P). 


On  March  31,  1991.  the  school  added  a  new  course  area  (Primary  Power  Distribution  and  Supplies)  and  defined  it  as  Area  13.  The  school 
renamed  previous  Areas  13  and  14  as  14  and  15.  Form  1  covers  students  entering  the  school  before  March  31.  1991.  Form  2  addresses 
students  entenng  the  school  after  this  date. 


13  513 


13  613 


14  514 


14  614 


15  515 

15  615 


Knowledge: 

Form  1:  WSC-3  UHF  transceiver,  functional  operation,  power  supply  &  distribution,  frequency  standard  &  synthesizer 
operation,  transmitter  function/signal  flow/conirol  operation /distribution/intcrfacc/BlTE  operation 
analysis,  reevr  operation/signal  flow/BITE  operation,  reevr  &  xmtr  paper  troubleshooting  (UHF-K). 

Form  2:  URT-23/URA-38  operation,  primary  power  circuit  analysis  (URT/URA-K). 

Performance: 

Form  1:  WSC-3  UHF  transceiver  operations  &  troubleshooting  (UHF-P). 

Form  2:  URT-23/URA-38  operations  &  troubleshooting  (URT/URA-P). 

Knowledge: 

Form  1:  Telecommunication  systems,  communications  planning,  blueprints,  remote  units,  teletype  principles  &  patch 
panels,  URA-17  teletype  converter  circuit  analysis,  (2V-2460  &  UCC-1  multiplexer  converter  &  keycr 
function  circuit  analysis  (Tclccom-K). 

Form  2i  WSC-3  UHF  transceiver,  functional  operation,  power  supply  &  distribution,  frequency  standard  &  synthesizer 
operation,  transmitter  function/signal  flow/control  opcration/distribution/inierface/BITE  operation 
analysis,  reevr  opcration/signal  flow/BITE  operation,  reevr  &  xmtr  paper  troubleshooting  (UHF-K). 

PeffbimaiKx: 

Form  1:  Telecommunication  systems  operations  &  troubleshooting  (Tciccom-P), 

Fcmn  2:  WSC-3  UHF  transceiver  operations  &  troubleshooting  (UHF-P). 

(Form  2  only)  Knowledge:  Telecommunication  systems,  communications  planning,  blueprints,  remote  units,  teletype 
principles  &  patch  panels,  URA-17  teletype  converter  circuit  analysis,  CV-2460  A  UCC-1  multiplexer 
converter  A  keycr  function  circuit  analysis  (Telecom-K). 

(Form  2  only)  Pexformanoe:  Telecommunication  systems  operations  A  troubleshooting  (Tclecom-P). 


*  K:  Knowledge,  P:  Performance. 
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Table  44  (continued ) 


\:ca  Test 


21  521 

22  ^22 

22  022 

23  523 

23  623 

24  524 

24  624 

2'  525 

25  625 


Dcscnpiion  (Name  -  Tspe  of  Test)® 


Knowledge:  Intro  to  radar  &  AN/SPS-10,  radar  principles,  intro  to  systems  vacuum  tube  principles,  microwave 
transmuting  tubes  (Radar-K). 

Knowledge:  V'acuum  lube  to  point  ground.  MPG  microwave  devices  &,  lubes,  radar  frequency  system  (MPG-K). 

Perfonnance:  Vacuum  tube  to  point  ground  &  MPG  microwave  devices  troubleshooting  labs  (MPG-P). 

Knowledge:  Intro  to  AN/SPS  receivers.  IF  strip,  dctcct/video.  interference  elimination  circuits,  AFC  synchro/scivos. 
antenna  svstems  fSPS-K). 

Performance:  Receiver  &  systems  labs  (SPS-P). 

Knowledge:  Planned  position  indicator  display,  intro  to  AN/SPA-25  &  special  circuits.  AN/SPA  timing  section,  power 
supply,  operation  &  timing  circuits  (SPAl-K). 

Performaoce:  AN7SPA-25  troubleshooting  (SPAl-P). 

Knowledge:  .*\N7SPA-25  azimmuth  resolver  &  bearing  cursor,  video  cursor  swttp  switch,  sweep  generators /yoke 
drivers/  range  strobe  generator  block,  range  nng  generator,  vidio  section  (SPA2-K). 

Perfonnance:  AN7SPA-25  sweep  &  brightening  troubleshooting  (SPA2-P). 


K:  Knowledge.  P:  Performance. 


Results  and  Discussion.  As  noted  above,  on  March  31,  1991,  the  school 
added  a  new  training  area  and  reduced  the  time  devoted  to  each  training  area. 
Because  of  these  changes,  RGI  separated  the  students  based  upon  whether  they 
entered  the  school  before  or  after  the  changes,  and  we  analyzed  the  two  forms 
separately.  Form  1  contains  students  who  entered  the  course  before  March  31, 
1991  (n  =  70).  Form  2  contains  students  taking  the  course  after  this  date  (n  = 
44).  Form  1  included  16  cases  with  missing  data.  RGI  excluded  these  cases  from 
some  of  the  analyses,  resulting  in  a  final  sample  of  54  cases  for  Form  1.  Form 
2  included  15  cases  with  missing  data.  Likewise,  we  excluded  these  cases  from 
some  of  the  analyses,  resulting  in  a  final  sample  size  of  29  cases  for  Form  2. 

RGI  computed  descriptive  statistics  for  each  of  the  test  variables.  Table 
45  presents  these  statistics.  Because  Form  2  contained  zero  cases  for  four  of 
the  variables  (URT/URA-K,  URT/URA-P,  UHF-K,  and  UHF-P),  we  did  not  include  these 
tests  in  the  analyses.  Also,  because  many  of  the  students  missed  several  of  the 
exams,  we  were  not  able  to  conduct  a  factor  analysis  for  this  school. 

Next,  we  sought  to  develop  composite  variables  which  would  capture 
essential  elements  of  the  criterion  variance  and  also  represent  meaningful 
content  dimensions  useful  to  the  ET  school.  In  order  to  examine  whether  test 
method  could  explain  aspects  of  school  performance,  we  developed  a  knowledge  test 
(KTests)  composite  and  a  performance  test  (PTests)  composite.  Since  the  ETl 
school  computes  the  FS6  using  63  as  the  minimum  score,  RGI  calculated,  its  own 
final  grade  (FSG2)  using  initial  test  scores.  We  used  the  school’s  weighting 
scheme  to  compute  FSG2:  (average  of  all  knowledge  tests  *  .7)  +  (average  of  all 
performance  tests  *  .3).  We  listed  both  FSG  and  FSG2  as  composite  variables. 
Table  46  presents  the  descriptive  statistics  for  each  of  the  composites  mentioned 
above.  We  placed  all  the  composite  variables  on  a  scale  of  0-100  to  compare  them 
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Tabic  45 


Descnnive  Statistics  for  ET2  \*anables 


Variable 

Mean 

Sid  Dev 

Skew 

Kurtosis 

Min 

Max 

Form  1: 

FM/DC-K 

69 

78.252 

12559 

-.872 

1.749 

33.30 

100.00 

SSB  Rccvt-K 

69 

77583 

15.155 

-1.188 

1501 

23.30 

96.70 

SSB  Rccvt-P 

69 

87.825 

16.655 

-2.137 

4.850 

22.00 

100.00 

SSB  Xmtr-K 

65 

85.640 

8541 

-565 

-.137 

6350 

100.00 

SSB  Xmtr-P 

65 

97,308 

19585 

-3.161 

9.382 

12.00 

100.00 

AN/WROK 

64 

80,627 

12.072 

-1.022 

.429 

50.00 

96.70 

AN/\\^RC-P 

64 

74.828 

31.109 

-1.119 

-  .455 

14.00 

100.00 

AN/URT-K 

63 

84.290 

9.304 

-.707 

.412 

56.70 

100.00 

.V</URT-P 

63 

87,746 

16.661 

-3,107 

11.381 

7.00 

100.00 

L’HF-K 

61 

81.639 

10.122 

-.908 

1.937 

43.30 

100.00 

UHF-P 

61 

84.410 

23.207 

-1.844 

2.489 

15.00 

100.00 

Telccom-K 

61 

*’7.872 

9.968 

-1.079 

2.161 

40.00 

93.30 

Tc!ccom-P 

61 

84.769 

22569 

-2,252 

4578 

10.00 

100.00 

Radar-K 

59 

83.344 

9.087 

-.878 

1.342 

53.30 

100.00 

MPG-K 

59 

74.244 

11.888 

-.462 

-.832 

46.70 

93.30 

MPG-P 

59 

79.669 

24.120 

-1.280 

.181 

24.00 

100.00 

SPS-K 

58 

73.102 

12.473 

-.340 

.094 

40.00 

100.00 

SPS-P 

58 

86.248 

19.761 

-2.944 

9.036 

3.00 

100.00 

SPAl-K 

57 

76.723 

11.171 

-.389 

-.001 

50.00 

100.00 

SPAl-P 

56 

81.259 

14.139 

-2501 

8.830 

13.00 

98.00 

SPA2-K 

55 

80.855 

10591 

-.427 

.453 

50.00 

100.00 

SPA2-P 

55 

79.727 

19.496 

-2.332 

6.222 

5.00 

9850 

Form  2: 

FM/DC-K 

44 

79.777 

11.689 

-.761 

.478 

50.00 

100.00 

SSB  ReevT-K 

43 

78.130 

15.964 

-1.217 

1517 

30.00 

96.70 

SSB  Reevr-P 

43 

91.698 

14.342 

-2.252 

4.155 

47.00 

100.00 

SSB  Xmtr-K 

43 

83.723 

9.121 

-.054 

-.912 

66.70 

100.00 

SSB  Xmtr-P 

43 

95,488 

10573 

-4.377 

21.729 

38.00 

100.00 

AN/WRC-K 

42 

81.824 

11.338 

-.692 

.018 

5350 

•100.00 

AN/WRC-P 

42 

82581 

27.048 

-2.096 

2.958 

3.00 

100.00 

AN/URT-K 

42 

85.871 

9.304 

-.431 

-  580 

66.70 

100.00 

AN/URT-P 

42 

89.952 

11.941 

-1.942 

4.279 

45.00 

100.00 

Tciccom-K 

31 

83526 

12.282 

-.758 

-.215 

56.00 

100.00 

Tclecom-P 

31 

92.103 

14.722 

-2.887 

8.719 

35.00 

100.00 

Radar-K 

40 

81.737 

9.426 

-.711 

.023 

60.00 

.  96.00 

MPG-K 

40 

78545 

11.478 

-.244 

-.933 

5350 

96.70 

MPG-P 

40 

86.030 

19.936 

-1.785 

1561 

35.00 

100.00 

SPS-K 

40 

78.000 

9.885 

.032 

-1.178 

60.00 

93.30 

SPS-P 

40 

75.000 

32564 

-1500 

-  .149 

350 

100.00 

SPAl-K 

40 

77.917 

13536 

-1.117 

1554 

36.70 

96.70 

SPAl-P 

40 

78.037 

20.798 

-1527 

1.317 

25.00 

9950 

SPA2-K 

39 

81.105 

9.088 

.091 

-.740 

6350 

96.70 

SPA2-P 

39 

83.979 

14.690 

-2.016 

4.862 

30.00 

100.00 

75 


Tabic  4f) 


Descnnive  Statistics  for  fcT2  Composite  Variables 


Composiic  n  Rel®  Mean  Std  Dev  Skew  Kurtosis  Min  Max 


Form  1: 

FSG 

FSG2*° 

KTesi^ 

PTcst^ 

Form  2: 


FSG 

44 

S/A 

81.891 

6.831 

.120 

•1.066 

70.30 

94.70 

FSG2 

29 

.96 

81.650 

7.296 

-  .245 

1.355 

68.10 

9SSO 

KTest 

29 

.96 

79.961 

7.476 

-  .368 

•  .511 

64.00 

92.91 

PTesr 

29 

.S7 

85^90 

9.437 

-  .583 

-  .386 

62.50 

98.17 

.'.ese  are  composites  based  on  communaiity  estimates  determined  through  regression  procedures. 
^  FSG2:  Final  School  Grade  computed  using  the  initial  test  scores. 

^  KTests:  Average  of  all  knowledge  test  scores. 

^  PTests:  Average  of  all  performance  test  scores. 


70 

N/A 

80394 

5.334 

.196 

-  .430 

68.70 

93.80 

54 

.91 

82,329 

5.668 

-  .315 

-  .145 

68.30 

93.70 

54 

.91 

81.005 

5.882 

-  .106 

-  .199 

68.93 

92.12 

55 

.72 

85.266 

8.907 

-1.123 

1.149 

57.70 

97.65 

more  readily.  The  sample  size  varied  for  the  composites  depending  on  the  number 
of  tests  missing  in  each  case. 

Finally,  we  computed  Pearson  correlation  coefficients  for  the  composites. 
Table  47  presents  the  composite  variable  correlations.  For  Form  2,  FSG  and  FSG2 
correlate  low  (.64),  considering  that  they  should  measure  the  same  thing.  For 
Form  1,  the  correlation  between  FSG  and  FSG2  is  somewhat  higher  (.87),  although 
this  form  contains  a  smaller  sample  size.  These  findings  support  using  initial 
test  scores  to  compute  a  final  school  grade  (FSG2). 

In  both  cases,  KTests  correlates  highly  with  FSG2  (.90  for  Form  1,  .95  for 
Form  2).  PTests  correlates  less  strongly  (.74  and  .82,  respectively).  Finally, 
the  correlation  between  KTests  and  PTests  is  .36  for  Form  1  and  .60  for  Form  2. 
Considering  that  the  sample  with  the  larger  n  (Form  1)  shows  a  lower  correlation, 
these  findings  suggest  that  the  composites  are  relatively  independent. 
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Tabic  47 


Correlations  Among  ET2  Comnosues 


Variable 

FSG 

FSG2» 

ICTests'^ 

PTests' 

FSG 

.87 

.88 

.62 

FSG2 

.64 

— 

.95 

.82 

K  Tests 

.61 

.90 

— 

.60 

P  Tests 

.41 

,74 

.36 

— 

Note .  Coefficients  below  the  diagonal  correspond  to  Form  1  (n  -  5A).  Coefficients  above  the  diagonal 
correspond  to  Form  2  (n  ■  29). 

®  FSG2:  Final  School  Grade  computed  using  the  initial  test  scores, 

^  KTests :  Average  of  all  knowledge  test  scores, 

^  PTests:  Average  of  all  performance  test  scores. 


Recommendations.  In  addition  to  FSG2,  use  PTests  (an  average  of  all 
performance  tests) . 
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Engineman  (EN) 

Description  of  Variables.  Before  October  1,  1990,  all  EN  students  took 
Propulsion  Engineering  (PE)  Basics  as  a  prerequisite  course.  Civilian  personnel 
taught  PE,  and  military  personnel  taught  EN.  As  of  October  1,  1990,  the  EN 
school  incorporated  the  PE  school  into  the  EN  course  curriculum  increasing  the 
length  of  the  school  from  9  to  11  1/2  weeks.  The  school  restructured  and 
reordered  the  presentation  of  the  previous  course  material  and  integrated  it  with 
the  new  PE  Basics  material.  However,  course  content  did  not  change.  Following 
this  merger,  teaching  responsibilities  for  the  PE  course  shifted  from  civilian 
to  mil itary  personnel . 

In  May  of  1991  (the  school  could  not  provide  a  more  specific  date),  the 
school  prepiloted  a  revised  course  curriculum  that  incorporated  several  new 
lessons.  At  the  time  of  this  revision  the  school  also  separated  the  curriculum 
into  3  distinct  phases.  Phase  I  covers  the  first  four  weeks  of  training,  Phase 
II  covers  weeks  five  through  eight,  and  Phase  III  covers  the  last  three  and  a 
half  weeks.  The  school  administers  a  comprehensive  phase  exam  following  each 
phase.  The  school  also  phased  out  quizzes  during  the  May  1991  prepilot  of  the 
new  three-phase  curriculum.  The  school  began  implementing  this  new  curriculum 
in  May  of  1991  and  finalized  it  in  October  of  1991. 

The  school  requires  a  score  of  75%  or  higher  to  pass  each  quiz  and  exam. 
Examinees  who  fail  a  quiz  or  exam  participate  in  remediation  and  retesting.  The 
instructor  assigns  the  minimum  passing  score  to  those  students  who  pass  the 
retest,  and  the  school  uses  that  minimum  score  in  calculating  the  final  school 
grade  (FSG).  All  students  who  fail  the  retest  appear  before  the  Academic  Review 
Board.  RGI  entered  only  the  initial  test  scores  for  each  examinee.  However,  the 
school  assigns  a  score  of  75  for  all  successful  retests. 
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As  a  result  of  these  revisions,  there  are  three  test  forms  for  the  EN 
database:  Test  Form:  A=Through  September  30,  1990;  B=October  1,  1990  through 
May  1991;  and  C=After  May  1991.  Table  49  presents  the  EN  test  numbers  and  a 
description  of  the  tests  followed  by  the  name  in  parentheses. 

To  compute  FSG,  the  EN  school  used  the  following  formulas  during  the 
various  Phases.  For  Phase  A  (Form  A),  the  school  weighted  quizzes  3.33%  (6 
quizzes  x  3.33%  =  20%),  exams  10%  (5  exams  x  10%  =  50%),  and  comprehensive  exams 
15%  (2  comprehensive  exams  x  15%  =  30%).  For  Phase  B  (Form  B),  the  school 
weighted  quizzes  1.75%  (8  quizzes  x  1.75%  =  14%),  exams  8.0%  (7  exams  x  8.0%  * 
56%),  and  comprehensive  exams  15%  (2  comprehensive  exams  x  15%  *  30%).  For  phase 
C  the  school  weighted  exams  6.875%  (8  exams  x  6.875%  =  55%)  and  comprehensive 
exams  15%  (3  comprehensive  exams  x  15%  =  45%). 

Sample.  The  Form  A  sample  consisted  of  137  students,  the  Form  B  sample' 
consisted  of  423  students,  and  the  Form  C  sample  consisted  of  300  students. 

Results  and  Discussion.  We  began  by  conducting  an  analysis  to  determine 
if  there  were  differences  among  the  three  groups  due  to  changes  in  the  school 
curriculum.  We  conducted  a  multivariate  analysis  of  variance  (MANOVA)  to  test 
the  covariance  matrices  of  five  test  scores  having  similar  content  across  the 
three  groups.  A  homogeneity  of  covariances  revealed  that  the  three  groups  had 
significantly  different  covariance  matrices  (F»6.688,  p<.001).  The  test  means 
also  differed  significantly  {F-17.72,  p<.001).  We  then  conducted  pairwise 
comparisons  and  found  significant  differences  in  the  covariance  matrices  (Groups 
A  and  B,  F*6.525,  p<.001;  Groups  A  and  C,  F*3.644,  p<.001;  Groups  B  and  C, 
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Tabic  4^ 


Descnrtions  cf  EN  V'ariables 


Week  lest  No.  Descnpuon  (Namc-Tv-pc  or  Test)^ 


Test  Fonn  A 

1  013  Quiz:  Introduction  to  diesel  engines,  diesel  engine  consirucuon.  diesel  engine  air  intake  systems,  diesel  engine  exhaust 

systems.  (Diesel  Engjnes-O) 

1  016  Rxam:  Introduction  to  diesel  engines,  diesel  engine  construction,  diesel  engine  air  intake  systems,  diesel  engine 

exhaust  systems.  (Diesel  Engines*K) 

2  023  Quit  Diesel  engine  fuel  systems,  diesel  engine  lube  oil  systems,  diesel  engine  cooling  water  systems.  (Diesel  Fuel 

Sys.-O) 

2  026  Exam;  Diesel  engine  fuel  systems,  diesel  engine  lube  oil  svstems.  diesel  engine  cooling  water  systems.  (Diesel  Fuel 

S>-s.-K) 

3  033  Quiz:  Diesel  engine  air  staning  systems,  diesel  engine  control  systems,  intro,  to  GM  6-71  scries  diesel  engine.  (Air 

Start  Sv's.-O) 

3  036  Fjam-  Diesel  engine  air  staning  systems,  diesel  engine  control  systems,  intro,  to  GM  6-71  scries  diesel  engine.  (Air 

Stan  Sy-s.-K) 

Quiz:  Clutches,  main  reduction  gear,  main  reduction  gear  lube  oil  system,  main  propulsion  shafting  system, 
controllable  pitch  propeller  system,  electro-hydraulic  steering  system.  (Clutches-Q) 

Comprehensive  Exam  1:  Dutches,  main  reduction  gear,  main  reduction  gear  lube  oil  system,  main  propulsion  shafting 
system,  controllable  pitch  propeller  system,  electro-hydraulic  steering  system,  all  material  covered  in  weeks  1  through 
5.  (Comprehensive  Testl) 

Quxe  Intro,  to  shipboard  electrical  circuits,  60Hz  electrical  distribution  system,  interior  communications  (IC)  alarm 
systems,  small  craft  operations,  small  craft  electrical  distribution  system,  small  craft  propulsion  engine  hydraulic 
starting  system,  small  craft  steering  systems,  small  craft  propulsion  engine  transmissions,  smalt  craft  ramp  hoisting 
system.  (Ship  Elcc.-Q) 

Intro,  to  shipboard  electrical  circuits,  60Hz  electrical  distribution  system,  interior  communications  (IC)  alarm 
systems,  small  craft  operations,  small  craft  electrical  distribution  system,  small  craft  propulsion  engine  hydraulic 
starting  system,  small  craft  steering  systems,  small  craft  propulsion  engine  transmissions,  small  craft  ramp  hoisting 
system.  (Ship  Elcc.-K) 

Quiz  Evaporators  (distilling  plants),  auxiliary  boilers,  air  conditioning  and  refrigeration  plants,  compressed  air 
systems,  shipboard  drainage  systems,  shipboard  fire  fighting  systems,  fuel  systems.  (Shpbd.  Fire  Fight-Q) 

Exam:  Evaporators  (distilling  plants),  auxiliary  boilers,  air  conditioning  and  refrigeration  plants,  compressed  air 
systems,  shipboard  drainage  systems,  shipboard  fire  fighting  systems,  fuel  systems.  (Shpbd.  Fire  Fight-K) 

Comprebeosive  Exam  2  Shipboard  auxiliary  equipment,  propulsion  plant  operations,  small  craft  operations,  all 
information  covered  in  weeks  6  through  9.  (Comprehensive  Tcst2) 


^Key  to  type  of  test:  K-Knowledge,  Q-Quiz 


5  053 

5  056 

6  063 

6  066 

7  073 

7  076 

9  800 
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Table  4b  (Continued  J 


Week  Test  No.  Descnpiion  tNamc-Tv-pc  of  Tcst)^ 

Test  Form  B 

1  013  Ouic  Command /dept. /watch  organization,  ships  maintenance  and  material  management  (5-M)  system,  equipment 

tag-out  procedures,  engineering  fundamentals,  personnel  qualification  standards  (PQS)  program,  engineering 
operational  sequencing  system  (EOSS).  (Eng.  Fundmtls.-O) 

1  016  PjtattL-  Command/dept./watch  organization,  ships  maintenance  and  material  management  (3-M)  system,  equipment 

tag-out  procedures,  engineering  fundamentals,  personnel  qualification  standards  (PQS)  program,  engineering 
operational  sequencing  system  (EOSS).  (Eng.  Fundmtls.-K) 

2  023  Quic  Technical  manuals,  common  hand  tools,  precision  measuring  instruments,  metal  fasteners,  pipe,  tubing  and 

fittings,  packing,  gaskets  and  insulation,  valves  (^obe.  gate  butterfly,  ball,  plug  cock,  needle).  (Tech.  Manuat-O) 

2  026  Pjam-  Technical  manuals,  common  hand  tools,  precision  measuring  instruments,  metal  fasteners,  pipe,  tubing  and 

fittings,  packing,  gaskets  and  insulation,  valves  (globe,  gate  butterfly,  ball,  plug  cock,  needle).  (Tech.  Manual-K) 

3  033  Ouir  Heat  exchangers,  lubricants,  bearings,  gears,  couplings,  pumps,  intro,  to  shipboard  electrical  circuits,  interior 

communication  alarms  system  60Hz  electrical  distribution  system,  shipboard  internal  communication  devices.  (Ship 
Elec. /Com  m-O) 

3  036  P-nm:  Heat  exchangers,  lubricants,  bearings,  gears,  couplings,  pumps,  intro,  to  shipboard  electrical  circuits,  intenor 

communication  alarms  system  60Hz  electrical  distribution  system,  shipboard  internal  communication  devices.  (Ship 
Elec./Comm-K) 

4  W3  Ouic  Compressed  air  systems,  auxiliary  machinery  cooling  water  system,  potable  water  system,  main  drain  system, 

waste  v^*ater/oily  waste  systems,  fire  fighting  equipment,  shipboard  firemain  system,  single  agent  fire  fighting  system, 
halon  1301  fire  fighting  system,  cold  iron  watchstation  indoarination.  (Shpbd.  Fire  Fight-Q) 

4  046  Compressed  air  systems,  auxiliary  machinery  cooling  water  system,  pouble  water  system,  main  drain  system. 

waste  water/oiiy  waste  systems,  fire  fighting  equipment,  shipboard  firemain  system,  single  agent  fire  fighting  system, 
halon  1301  fire  fighting  system,  cold  iron  watchstation  indoctrination.  (Shpbd.  Fire  Fight-K) 

5  053  Quic  Diesel  engine  construction,  diesel  engine  air  intake  systems,  diesel  engine  exhaust  systems,  diesel  engine  cooling 

water  systems.  (Diesel  Engjnes-Q) 

5  056  Compxebettsivc  Paam  1:  Diesel  engine  construction,  diesel  engine  air  intake  systems,  diesel  engine  exhaust  systems, 

diesel  engine  cooling  water  systems,  all  material  covered  in  weeks  1  through  5.  (Comprehensive  Testl) 

7  073  Quiz:  Diesel  engine  lube  oil  systems,  diesel  engine  fuel  systems  (external  systems),  diesel  engine  fuel  systems 

(injection  systems),  diesel  engine  control  systems,  Diesel  engine  air  suning  systems,  lube  oil  fill,  transfer  and 
purification  system,  fuel  oil  fill,  transfer  and  purification,  stripping  and  ballast  systems.  (Fuel/Start  Sys.-Q) 

7  076  Exam:  Diesel  engine  lube  oil  systems,  diesel  engine  fuel  systems  (external  systems),  diesel  engine  fuel  systems 

(injection  systems),  diesel  engine  control  systems,  Diesel  engine  air  starting  systems,  lube  oil  fill,  transfer  and 
purification  system,  fuel  oil,  purification,  transfer,  stripping  and  ballast  systems.  (Fuel/Start  Sys.-K) 

8  083  Quiz:  Intro,  to  diesel  engine  maintenance  (troubleshooting,  preparation  for  overhaul,  disassembly,  precision 

measuring,  reassembly).  (Engine  Maint.T/S<0) 

8  086  Exam*  Intro,  to  diesel  engine  maintenance  (troubleshooting,  preparation  for  overhaul,  disassembly,  precision 

measuring,  reassembly).  (Engine  Maint.T/S-K) 

9  093  Quic  Qutches.  main  reduction  gears,  main  reduction  gear  lube  oil  system,  pain  propulsion  shafting  system,  basic 

hydraulics.  (Qutches-Q) 

9  096  Exam  Qutches.  main  reduction  gears,  main  reduction  gear  lube  oil  system,  main  propulsion  shafting  system,  basic 

hydraulics.  (Qutches-K) 

*Kcy  to  Type  of  Test:  K-Knowtedge,  Q-Quiz 
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Tabic  4S  fConimucdi 


l  est  No.  Dcscnpiion  ( Namc-T>pc  of  Test 


100  ComprcbcDsh^  Eiain  3:  Auxiliary  equipment,  evaporators  (distilling  plants),  auxiliary  boilers,  air  conditioning  and 
refrigeration,  small  boat  electrical  distribution  system,  small  boat  hydraulic  starting  system,  small  boat  steering  system, 
small  boat  transmissions,  small  boat  bow  ramp  hoisting  systems,  small  boat  operations,  propulsion  plant  operations, 
all  material  covered  in  weeks  6  through  11  1/2.  f Comprehensive  Tcst2) 

Test  Fonn  C 

016  Ham:  Command/dept. /watch  organization,  intro,  to  engineering  programs,  shipboard  safety  programs,  electrical 
safety  program,  hearing  conservation  program,  heat  stress  program,  hazardous  material  program,  environmental 
protection,  ships  maintenance  and  material  management  (3»M)  system,  equipment  tag*out  procedures,  engineering 
fundamentals,  personnel  qualification  standards  (POS)  program,  engineering  operational  sequencing  system  (BOSS). 
(Eng.  Fundmtis.-K) 

026  Exam:  Technical  manuals,  common  hand  tools,  precision  measuring  instruments,  metal  fasteners,  pipe  tubing  and 
fittings,  packing,  gaskets  and  insulation,  valves  (globe,  gate  butterfly,  ball,  plug  cock,  needle).  (Tech,  Manual-K) 

036  Fxam’  Heat  exchangers,  lubricants,  bearings,  gears,  couplings,  pumps,  intro,  to  shipboard  electrical  circuits,  interior 
communication  alarms  system.  60Hz  electrical  disinbution  s\'stcm.  shipboard  internal  communication  devices.  (Ship 
Elcc./Comm-K) 

046  ComprebensnfC  Rnm  1:  Compressed  air  system,  auxiliary  machinery  cooling  water  system,  potable  water  system,  main 
drain  system,  waste  water/oily  waste  systems,  fire  fighting  equipment,  shipboard  firemain  system,  single  agent  fire 
fighting  system,  halon  1301  fire  fighting  system,  cold  iron  watchstation  indoctrination,  all  material  covered  in  weeks 
1  through  4.  (Comprehensive  Tcstl) 

056  Exam*  Diesel  engine  construction,  diesel  engine  air  intake  systems,  diesel  engine  exhaust  systems,  diesel  engine 
cooling  water  systems.  (Diesel  Engines-K) 

066  Exam:  Diesel  engine  lube  oil  systems,  diesel  engine  fuel  systems  (external  systems),  diesel  engine  fuel  systems 
(injection  systems),  diesel  engine  control  systems.  (Fucl/Siart  Sys.-K) 

076  Paam*  Diesel  engine  air  starting  systems,  surface  ship  noise  awareness  program,  lube  oil  quality  management 
program,  lube  oil  fill,  transfer  and  purification  system,  fuel  oil  fill,  transfer  and  purification,  stripping  and  ballast 
systems.  (Air  Start  Sys-K) 

Comprcbciisn^  Exam  ^  Intro,  to  diesel  engine  maintenance  (troubleshooting,  preparation  for  overhaul,  disassembly, 
precision  measuring,  reassembly),  all  material  covered  in  weeks  5  through  8.  (Comprehensive  Tcst2) 

096  f^aam*  Qutches,  main  reduction  gears,  main  reduction  gear  lube  oil  system,  main  propulsion  shafting  systemf  basic 
hydraulics.  (Qutches  >  K) 

106  Exam:  Shipboard  auxiliary  equipment,  evaporators  (distilling  plants),  auxiliary  boilers,  air  conditioning  and 
refrigeration,  small  boat  electrical  distribution  system,  small  boat  hydraulic  starting  system,  small  boat  steering  system, 
small  boat  transmission  system,  small  boat  bow  ramp  hoisting  systems,  small  boat  operations.  (Ship  Aux.Equip/£vap6- 
K) 

116  Eam/ComprebcnsM  Exam  3:  Propulsion  plant  operation  and  all  material  covered  in  weeks  9  through  11  1/2. 
(Comprehensive  Test3) 


*Kcy  to  Type  of  Test:  K-Knowlcdgc,  Q-Quiz 
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F=8.410,  p<.001)  and  the  means  (Groups  A  and  B,  F*9.382,  p<.001;  Groups  A  and  C, 
F*13.41,  p<.001;  Groups  B  and  C,  F=29.328,  p<.001)  for  all  pairwise  comparisons. 

To  test  whether  ability  accounted  for  these  differences  between  the  three 
groups,  we  conducted  a  multivariate  analysis  of  covariance  (MANCOVA)  using  the 
students’  pre-enlistment  ASVAB  scores  as  covariates.  After  partialling  out  the 
variance  due  to  ASVAB  scores,  the  test  for  homogeneity  of  covariance  matrices 
revealed  different  matrices  for  the  three  groups  (F»1.886,  p<.001).  The  means 
also  differed  across  the  three  groups  after  accounting  for  variance  due  to  the 
ASVAB  scores  (F=16.012,  p<.001).  We  then  conducted  pairwise  comparisons  and 
found  significant  differences  in  the  covariance  matrices  (Groups  A  and  B, 
F=1.821,  p<.001;  Groups  A  and  C,  F=1.679,  p<.001;  Groups  B  and  C,  F=2.029, 
p<.001)  and  the  means  (Groups  A  and  3,  F*10.349,  p<.001;  Groups  A  and  C, 
F»11.267,  p<.001;  Groups  B  and  C,  F'24.377,  p<,001)  for  all  pairwise  comparisons. 
These  results  suggest  that  the  no  d  differences  among  the  three  groups  are  due 
to  something  other  than  differences  in  ability,  and  they  indicate  that  the  three 
samples  come  from  three  distinct  populations. 

To  further  evaluate  the  conclusion  of  differences  among  groups,  we  computed 
effect  sizes  (ES)  as  measures  of  the  practical  significance  of  the  differences 
between  the  adjusted  means  across  groups.  Cohen  and  Cohen  (1977)  state  that  .2 
represents  a  small  ES,  .5  is  medium,  and  .8  is  large.  For  the  total  of  15 
comparisons  (5  variables  x  3  pairwise  comparisons),  we  found  3  ESs  within  the  .00 
and  .19  range,  8  ESs  within  the  .20  and  .49  range,  and  4  ESs  within  the  .50  and 
.79  range.  These  results  show  that  73%  of  the  ESs  fall  below  a  value  of  .50. 
Thus,  while  the  MANOVA  tests  indicate  that  the  groups  differ  from  one  another  in 
terms  of  statistical  significance,  the  ESs  suggest  small  magnitudes  of 
differences,  in  terms  of  practical  significance. 
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We  present  the  results  for  each  of  the  three  forms  separately.  A 
discussion  following  the  results  of  the  final  form,  however,  includes  all  three 
forms . 

Table  49  presents  the  descriptive  statistics  for  the  set  of  variables  used 
in  the  analysis  of  Form  A.  We  excluded  quizzes  from  the  factor  analysis  because 
there  were  no  quizzes  in  Form  C  and,  for  a  comparison,  we  wanted  a  uniform  set 
of  variables  across  forms.  We  conducted  a  factor  analysis  on  the  set  of 
knowledge  and  comprehensive  tests.  The  scree  test  and  factor  loadings  indicated 
a  one-factor  solution. 

Table  50  shows  the  commonalities,  factor  loadings,  and  factor  score 
coefficients.  All  tests  loaded  highly  on  the  factor,  with  factor  loadings 
ranging  from  .55  to  .80.  We  sought  to  develop  composite  variables  which  would 
capture  essential  elements  of  the  criterion  variance  and  also  represent 


Table  49 


Descriptive  Statistics  for  EN  Variables  -  Form  A 


Variable 

Mean 

Std  Dev 

Skewness 

Kunosis 

Minimum 

.Maxim 

Diesel  EngincsO 

143 

87.800 

10^55 

-1.445 

2.755 

40.00 

100.00 

Diesel  Engincs-K 

143 

84.322 

10.125 

-0.852 

0.915 

44.00 

100,00 

Diesel  Fuel  Sys.-Q 

142 

87.972 

9.664 

-0.861 

0.810 

56.00 

100.00 

Diesel  Fuel  Sv’s.-K 

142 

83.451 

9.726 

-0.899 

1.108 

52.00 

100.00 

Air  Start  Sys.^ 

140 

89.286 

8.104 

-0.732 

0.115 

64.00 

100.00 

Air  Stan  Sys.-K 

140 

83.471 

10.511 

-0.606 

0.117 

50.00 

100.00 

Qutches-Q 

140 

86.371 

11.231 

-0.817 

0.226 

52.00 

100.00 

Comprehensive  Testl 

140 

82.836 

9.078 

-0.088 

0.645 

62.00 

100.00 

Ship  Elcc.-Q 

138 

91.681 

8.372 

-1.168 

1.038 

64.00 

100.00 

Ship  Elcc.-K 

138 

88.261 

7.777 

-0.627 

0.044 

62.00 

100.00 

Shpbd  Fire  Fight.-Q 

138 

87^94 

9.825 

-1.336 

3.025 

44.00 

100.00 

Shpbd  Fire  Fight.-K 

138 

87.000 

8.3S4 

-0J40 

-0.066 

60.00 

100.00 

Comprehensive  Test2 

137 

81J84 

8.334 

-0.068 

-OJll 

60.00 

100.00 

Table  5'"' 

Communaliiies.  Facior  l  oading;  and  Factor  Score  Coefficients  for  EN  Variables  -  Form^ 


Variable 


Communality  Factor  Loadinp  Faaor  Score  Coefficients 


Diesel  Encincs-K 
Diesel  Fuel  Sv-s.-K 
Air  Sian  Sn-s.-K 
Comprehensive  Tcsil 
Ship  Elec.-K 
Shpbd  Fire  Fight.-K 
Comprehensive  Tesi2 


Note,  n  =  137 


.434  .690  158 

,487  .734  -194 

.293  SSS  103 

.475  .710 

.480  .723  -I'll 

.421  .673  149 


^69  .808 


meaningful  content  dimensions  useful  to  the  EN  school.  We  computed  composite 
variables  based  on  the  two  phases  in  the  curriculum.  Phase  1  included  all  tests 
before  comprehensive  test  and  including  comprehensive  test  1.  Phase  2  included 
all  tests  before  comprehensive  test  2  and  including  comprehensive  test  2.  We 
also  computed  a  composite  variable  based  on  all  of  the  Quizzes.  And  finally, 
because  the  EN  school  computed  the  FSG  using  a  75  as  the  minimum  score,  we 
calculated  our  own  Final  School  Grade  (FSG2)  using  the  initial  test  scores,  and 
listed  both  as  composite  variables.  Table  51  presents  the  descriptive  statistics 
for  the  composite  variables. 

Next,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  52  presents  the  composite  variable  intercorrelations.  Because 
FSG  and  FSG2  correlate  .97,  this  offers  no  support  for  using  initial  test  scores 
to  compute  FSG. 

FSG  correlates  highest  (.97)  with  Factor  1,  followed  closely  by  Phase  2 
(.94)  and  Phase  1  (.93).  Finally,  Phase  1  and  Phase  2  correlate  .92  and  .90  with 
FSG,  respectively.  This  supports  using  FSG  as  an  indicator  of  Factor  1. 


85 


rablc  51 


Descnrtive  Statistics  for  EN  Comtxpsite  Variables  -  Form  A 


Composite 

ji 

Rel« 

Mean 

Sid  Dev 

Skewness 

Kurtosis 

Minimum 

Maximum 

FSG“ 

143 

N/A 

85i54 

5.686 

0.031 

^.458 

67.10 

97.90 

FSG2^ 

137 

.892 

85.927 

6.182 

4).082 

-1.026 

73.14 

97.79 

Phase  1® 

140 

.777 

83.627 

7.480 

-0.160 

-0.853 

67.75 

98.00 

Phase  2^ 

137 

.766 

85.647 

6.691 

-0.063 

-0.981 

72.67 

99.00 

Quizzes^ 

137 

.682 

88.603 

5.912 

-0.300 

-0.816 

74.00 

100.00 

®Thcse  arc  composites  based  on  communality  csiimaies  and  the  procedures  described  in  the  text. 
^FSG;  Final  School  Grade  computed  by  the  EN  school. 

®FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

'^Phasel:  Average  of  all  tests  up  to  and  including  Comprehensive  Testl. 

®Phasc2:  Average  of  all  tests  up  to  and  including  Comprehensive  Tesi2. 

^Quizzes:  Average  of  all  Quizzes. 


Table  52 


Correlations  Among  EN  Composites  and  Factors  -  Form  A 


Composite 

Faciorl^ 

FSG° 

FSG2' 

Phase 

Phase  2« 

FSG 

.97 

FSG2 

.97 

.97 

Phase  1 

.93 

.92 

.96 

Phase  2 

.94 

.90 

.89 

.77 

Quizzes^ 

.76 

.83 

,87 

.77 

.68 

^These  are  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^FSG:  Final  School  Grade  computed  by  the  EN  school. 

^FSG2:  Final  School  Grade  computed  using  initial  scores. 

'^Phasel:  Average  of  all  tests  up  to  and  including  Comprehensive  Testl. 

®Phasc2:  Average  of  all  tests  up  to  and  including  Comprehensive  Test2. 

^Quizzes:  Average  of  all  Quizzes. 


Results  Form  B.  Table  53  presents  the  descriptive  statistics  for  the  set 
of  variables  used  in  the  analysis  of  Form  B.  We  excluded  quizzes  from  the  factor 
analysis  because  there  were  no  quizzes  in  Form  C  and,  for  a  comparison,  we  wanted 
a  uniform  set  of  variables  across  forms.  We  conducted  a  factor  analysis  on  the 
set  of  knowledge  and  comprehensive  tests.  The  scree  test  and  factor  loadings 
indicated  a  one-factor  solution. 
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Table  54  shows  the  communal i ties,  factor  loadings,  and  factor  score 
coefficients.  All  tests  loaded  highly  on  the  factor,  with  factor  loadings 
ranging  from  ,57  to  .77. 

We  sought  to  develop  composite  variables  which  would  capture  essential 
elements  of  the  criterion  variance  and  also  represent  meaningful  content 
dimensions  useful  to  the  EN  school.  We  computed  composite  variables  based  on  the 
two  phases  in  the  curriculum.  Phase  1  included  tests  before  comprehensive  test 
1  and  including  comprehensive  test  1.  Phase  2  consisted  of  all  tests  before 
comprehensive  test  2  and  including  comprehensive  test  2.  We  also  computed  a 
composite  variable  based  on  the  Propulsion  Engineering  (PE)  courses  and  a 
composite  based  on  all  of  the  Quizzes.  And  finally,  because  the  EN  school 
computed  the  FSG  using  a  75  as  the  minimum  score,  we  calculated  our  own  Final 
School  Grade  (FSG2)  using  the  initial  test  scores,  and  listed  both  as  composite 
variables.  Table  55  presents  the  descriptive  statistics  for  the  composite 
variables. 


Table  53 


Descnptive  Statistics  for  EN  Variables  -  Form  B 


Variable 

t\ 

Mean 

Sid  Dev 

Skewness 

Kurtosis 

Minimum 

Maximum 

Eng.  Fundmtls-Q 

469 

85.962 

-1.022 

2528 

24.00 

100.00 

Eng.  Fundmtis-K 

467 

82.107 

10.011 

-0574 

-0.085 

50.00 

100.00 

Tech.  Manual-Q 

459 

82.885 

11576 

-0.826 

0.824 

36.00 

100.00 

Tech,  ManuaUK 

457 

87.379 

8.064 

-0.678 

0.461 

54.00 

100.00 

Ship  EIcc./Comm.-O 

452 

89.212 

8.680 

-1.046 

0.768 

56.00 

100.00 

Ship  Dec./Comm.-K 

453 

83.907 

9548 

•0.683 

0.836 

44.00  * 

100.00 

Shpbd.  Fire  Fight-Q 

448 

83.679 

10580 

-0.650 

0.347 

48.00 

100.00 

Shpbd.  Fire  Fight-K 

448 

79.415 

11.243 

-1.151 

2.452 

22.00 

100.00 

Diesel  £ngines-Q 

440 

86.327 

10.028 

-0.962 

0.955 

48.00 

100.00 

Comprehensive  Testl 

438 

84.795 

6.657 

-0.240 

-0.216 

64.00 

100.00 

Fucl/Start.Sys.-0 

437 

85876 

11J283 

-0.471 

1.395 

40.00 

100.00 

FueI/Start.Sys.-K 

436 

80.128 

10.030 

-0.471 

-0.205 

48.00 

100.00 

Eng.  Maint.  T/S-Q 

430 

85851 

10.611 

-0.877 

0.660 

48.00 

100.00 

Eng.  Maim.  T/S-K 

429 

86.615 

7.772 

-1.037 

.  2.418 

44.00 

lOO.UO 

Qutches-O 

426 

86.873 

10.786 

-1.229 

2.286 

32.00 

100.00 

Qutches-K 

430 

81595 

8.816 

-0551 

0.295 

50.00 

100.00 

Comprehensive  Test3 

425 

78581 

8.452 

-0.616 

0.988 

41.00 

100.00 
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Tubic  5-i 


Communahnes.  Factor  i^adings  and  Factor  Score  Coefficients  for  EN  Variables  ■  Form  B 


Communaiirs- 

Factor  Loadings 

Factor  .'*^orc  Coefficients 

Eng.  Fundmtls.-K 

.303 

.586 

.130 

Tech  Manuai-K 

.365 

.639 

.143 

Ship  Elcc,/Comm.-K 

.363 

.648 

.156 

Shpbd.  Fire  Fight-K 

.362 

.603 

.117 

Comprehensive  Testl 

^25 

.772 

.268 

Fuel/Stan. Svs.-K 

.304 

.566 

.123 

Eng.  Maint.  T/S-K 

.348 

.610 

.139 

Cluichcs-K 

M2 

.571 

.119 

Comprehensive  Tcst2 

.355 

.635 

.146 

Note.  n  =  423 

Table  55 

Descnptjvc  Statistics  for  EN  Comoosite  Variables  - 

Form  B 

Composite  n 

Rel** 

.Mean 

Sid  Dev 

Skewness 

Kurtosis 

.Minimum 

.Maximum 

F^  •  469 

N/A 

83-522 

-2.177 

22.000 

24.00 

97.80 

FSU2=  422 

.877 

83.918 

5.706 

0.077 

-0.479 

68.02 

97.89 

Phase  1®  437 

,789 

83.934 

6-583 

-0.190 

-0.259 

61.20 

98.40 

Phase  2-  424 

,708 

82.468 

5.976 

-0.123 

0.078 

62.80 

97.20 

PE*  ^  453 

.668 

7.209 

-0.320 

-0.376 

60.67 

99.33 

Quizzes**  426 

.736 

6.023 

-0.369 

0.073 

6630 

99.60 

"FSG:  Final  School  Grade  computed  by  the  EN  school. 

^FSG2:  Final  School  Grade  computed  using  initial  test  scores, 

'^Phasel:  Average  of  all  tests  up  to  and  including  Comprehensive  Testl. 
®Phase2;  Average  of  all  tests  up  to  and  including  Comprehensive  Test2. 
^PE:  Average  of  all  Propulsion  Engineenng  tests  (first  3  tests). 
^Quizzes:  Average  of  all  Quizzes. 


Next,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  56  presents  the  composite  variable  intercorrelations.  Because 
FS6  and  FSG2  correlate  .97,  this  offers  no  support  for  using  initial  test  scores 
to  compute  FSG. 

FSG  correlates  highest  (.98)  with  Factor  1,  followed  by  Phase  1  (.93), 
Phase  2  (.92),  and  PE  (.86).  Finally,  Phase  1  and  Phase  2  each  correlate  .91 
with  FSG,  while  PE  correlates  .84  with  FSG.  This  supports  using  FSG  as  an 
indicator  of  Factor  1. 
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Tabic  5o 


Correlations  Among  EN  Composites  and  Factors  -  Form  B 


Composiic 

Factor  i 

FSG'^ 

FSG2^ 

Phase 

Phase  2" 

FSG 

.98 

FSG2 

.98 

.97 

Phase  1 

.93 

.91 

.92 

Phase  2 

.92 

.91 

.91 

.74 

PE 

.86 

.BA 

.87 

.93 

.67 

Quizzes* 

.80 

.83 

.87 

.74 

.75 

^FSG:  Final  School  Grade  computed  by  the  EN  school. 

^FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

^  Phase  1:  Average  of  all  tests  up  to  and  including  Comprehensive  Testl. 
^Phasc2:  Average  of  all  tests  up  to  and  including  Comprehensive  Tcsi2. 
®PE:  Average  of  all  Propulsion  Engincenng  tests  (first  3  tests). 
"Quizzes:  Average  of  all  Quizzes. 


Results  Form  C.  Table  57  presents  the  descriptive  statistics  for 
the  set  of  variables  used  in  the  analysis  of  Form  C.  We  conducted  a  factor 
analysis  on  the  set  of  knowledge  and  comprehensive  tests.  The  scree  test  and 
factor  loadings  indicated  a  one-factor  solution.  Table  58  shows  the 
communal ities,  factor  loadings,  and  factor  score  coefficients.  Factor  loadings 
ranged  from  .43  to  .76. 

We  sought  to  develop  composite  variables  which  would  capture 
essential  elements  of  the  criterion  variance  and  also  represent  meaningful 
content  dimensions  useful  to  the  EN  school.  We  computed  composite  variables 
based  on  the  three  phases  in  the  curriculum.  Phase  1  included  tests  before 
comprehensive  test  1  and  including  comprehensive  test  1.  Phase  2  consisted  of 
all  tests  before  comprehensive  test  2  and  including  comprehensive  test  2.  Phase 
3  consisted  of  all  tests  before  comprehensive  test  3  and  including  comprehensive 
test  3.  We  also  computed  a  composite  variable  based  on  the  Propulsion 
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1  able  57 


Degcnrtive  Statistics  for  EN  Vanahles  -  Form  C 


\  anabic 


Mean  itd  Dev  ikewness  Kunosis  Minimum  Maximum 


Eng.  Fundmils.-K 
Tech.  Manual-K 
Ship  EIec./Comm.*K 
Comprehensive  Tcstl 
Diesel  Engincs-K 
FueI/Start.S\'s.-K 
Air  Start  S^.-K 
Comprehensive  Tcsi2 
Clutches-K 

S  hipj^ux.  Equip/ Eva  ps-K 
Comprehensive  Test3 


40S 

S1.015 

9.395 

4).495 

399 

84.887 

8.647 

-0-517 

397 

86.272 

9.641 

-0.826 

395 

S2S22 

9,619 

-0.952 

307 

89.717 

7.074 

-1.255 

388 

84i88 

9.130 

-0546 

387 

85.824 

7.640 

-0.620 

3W 

85.957 

6.373 

-0.275 

386 

84.288 

10.013 

-0.881 

384 

87.764 

6.943 

-0.653 

300 

84.430 

7.047 

■0.477 

0.430 

40.00 

100.00 

0.046 

54.00 

100.00 

0577 

48.00 

100.00 

3.229 

23.00 

100.00 

3.070 

50.00 

100.00 

-0.074 

56.00 

100.00 

0.313 

58.00 

100.00 

-0.109 

64.00 

100.00 

-0.880 

46.00 

100.00 

0.623 

62.00 

100.00 

0.148 

57.00 

100.00 

Table  55 

Cpmmunaiities.  Factor  Loadings  and  Factor  Score  Coefficients  for  F.\  Variables  »  Form  C 


Variable 


Communality  Factor  Loadings  Factor  Score  Coefficients 


Eng.  Fundmtls.-K 

.214 

Tech.  Manual-K 

.301 

Ship  EIcc./Comm.-K 

.409 

Comprehensive  Testl 

.437 

Diesel  Engines-K 

.334 

Fuel/Stan.Sys.-K 

.390 

Air  Start  Sys.-K 

.389 

Comprehensive  Test2 

555 

CIutches-K 

.431 

ShipAux.Equip/E\'aps-K 

.310 

Comprehensive  Tcst3 

.483 

.429 

.053 

558 

,089 

.658 

.136 

.690 

.157 

572 

.084 

.640 

.112 

.640 

.127 

.759 

.208 

.660 

.120 

552 

.082 

.739 

.181 

Note,  ns 300 


Engineering  (PE)  exams.  And  finally,  because  the  EN  school  computed  the  FSG 
using  a  75  as  the  minimum  score,  we  calculated  our  own  Final  School  Grade  (FSG2) 
using  the  initial  test  scores,  and  listed  both  as  composite  variables.  Table  59 
presents  the  descriptive  statistics  for  the  composite  variables. 

Next,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  60  presents  the  composite  variable  intercorrelations.  Because 
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Tabic  59 


Descnnive  Statistics  for  EN  Composite  Variables  -  Form  C 


Composite 

Rel® 

.Mean 

Sid  Dev 

Skewness 

Kunosis 

.Minimum 

Maximum 

408 

N/A 

84J83 

5.994 

-1.458 

8.800 

40.00 

96.70 

FSG2® 

300 

.873 

85364 

5.353 

-0.126 

-0.445 

6930 

96.80 

Phase  1® 

395 

.698 

83.835 

6.745 

-0.405 

0.062 

58.75 

97.00 

Phase  2^ 

304 

.754 

86.762 

5.6S6 

-0.273 

-0.219 

68.25 

99.00 

Phase  3*^ 

300 

.699 

85.891 

6.175 

-0.423 

0.089 

67.00 

99.33 

PE*’ 

397 

.586 

84.238 

6.921 

-0.435 

-0.062 

63.33 

98.00 

®Thcsc  arc  composites  based  on  communaljt>'  estimates  and  the  procedures  described  in  the  text. 
^FSG:  Final  School  Grade  computed  by  the  EN  school. 

^FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

^Phasel:  Average  of  all  tests  up  to  and  including  Comprehensive  Testl. 

®Phase2:  Average  of  all  tests  up  to  and  including  Comprehensive  Test2. 

*Phase3:  Average  of  all  tests  up  to  and  including  Comprehensive  Test3. 

®PE:  Average  of  all  Propulsion  Engineering  tests  (first  3  tests). 


Table  60 

Correlations  Among  EN  Composites  and  Factors  -  Form  C 


Composite 

Factorl 

FSG® 

FSG2^ 

Phase  I'’ 

Phase  2"^ 

Phase3^ 

FSG 

.98 

FSG2 

.99 

.98 

Phase  1 

.84 

.85 

.88 

Phase  2 

.90 

.87 

.89 

.65 

Phase  3 

.87 

.85 

.85 

.62 

.70 

PE^ 

.78 

.79 

.84 

.96 

.60 

37 

®FSG:  Final  School  Grade  computed  by  the  EN  school. 

^FSG2:  Final  School  Grade  computed  using  initial  test  scores. 

^  Phase  1:  Average  of  all  tests  up  to  and  including  Comprehensive  Testl. 

°Phase2:  Avxrage  of  all  tests  up  to  and  including  Comprehensive  Test2. 

*Phasc3:  Average  of  all  tests  up  to  and  including  Comprehensive  Tcst3. 

^PE:  Average  of  all  Propulsion  Engineering  tests  (first  3  tests). 


FSG  and  FSG2  correlate  .98,  this  offers  no  support  for  using  initial  test  scores 
to  compute  FSG. 

FSG  correlates  .98  with  Factor  1,  followed  by  Phase  2  (.90),  Phase  3 
(.87),  Phase  3  (.84),  and  PE  (.78).  Finally,  Phase  1,  Phase  2,  and  Phase  3 
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correlate  .85,  .37.  and  .85  with  FSG,  respectively.  PE  correlates  .79  with  FSG. 
This  suoports  using  FSG  as  an  indicator  of  Factor  1. 

In  summary,  due  to  changes  in  the  course  curriculum  and  type  of 
instruction,  we  separated  the  EN  school  into  three  forms.  Based  on  the  results, 
however,  we  found  the  school  performance  measures  to  be  unidimensional  for  each 
form.  Results  for  each  of  the  three  forms  also  supports  using  the  FSG  as  an 
indicator  of  performance  in  the  school. 

Recommendations.  We  recommend  using  the  FSG  as  an  indicator  of  performance 


in  the  EN  school , 


Fire  Control  (FC) 

Description  of  Variables.  The  FC  school  provided  hard-card  data  for 
students  tested  through  September  1990  and  the  ISS  Response  History  Report  for 
students  tested  after  September  16,  1990.  The  data  format  variable  in  the 
database  distinguishes  between  the  two  types  of  student  data. 

The  hard-card  data  does  not  always  have  the  class  convening  date  on  the 
forms.  Therefore,  RGI  entered  the  first  test  date  (Julian  date  90101  -  90240) 
rather  than  the  class  convening  date  for  students  with  hard-card  data,  except 
classes  91277  and  91283,  which  were  pilot  classes.  RGI  entered  the  class 
registration  date  (Julian  date  90000  -  92366)  for  students  with  ISS  data  and  for 

classes  91277  and  91283. 

The  school  uses  a  multiple  choice,  written  test  format  and  administers 
tests  only  on  Fridays.  Therefore,  some  students  may  not  take  all  tests  for 
various  reasons  (such  as  when  a  government  holiday  falls  on  a  Friday).  The 
school  requires  a  70%  to  pass  each  test.  Students  who  fail  a  test  must  take  a 
retest  over  the  problem  areas  after  remediation.  The  school  allows  two  retests 
per  test.  Students  who  fail  both  retests  must  appear  before  the  Academic  Review 
Board.  As  of  June  1991,  the  school  administers  retests  on  Saturday  mornings 
instead  of  later  on  the  same  day  of  testing  (Friday).  According  to  instructors, 
this  extra  time  for  studying  greatly  improved  retest  scores.  The  school  assigns 
the  minimum  passing  score  to  all  retests  the  student  passes.  However,  RGI 

entered  the  initial  score  for  each  test. 

We  excluded  Modules  601  and  602  from  the  analyses,  because  we  could  not 
determine  whether  a  student  received  an  open  or  closed-book  test.  The  school 
left  the  method  of  testing  to  the  discretion  of  the  instructors.  We  also 
excluded  both  of  the  Control  Systems  tests  (CSl  and  CS2)  and  the  Fire  Control 
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Problem  (FCP)  test.  Only  25  students  took  these  tests  and  all  25  students  were 
missing  the  registration  Julian  date. 

The  school  calculates  the  Final  School  Grade  by  adding  all  of  the  module 
test  scores  together  and  dividing  the  total  by  the  number  of  tests  administered 
to  the  student. 

The  school  made  several  changes  to  the  curriculum  during  the  time  RGI 
collected  data.  These  changes  involved  renumbering  tests,  consolidating  and 
splitting  test  material,  and  adding  and  subtracting  tests  and  test  material. 
Table  61  describes  the  tests  and  curriculum  changes.  Notes  in  Table  61  indicate 
when  and  where  these  changes  occurred. 

For  each  test  variable,  Table  61  identifies  the  ISS  name  (e.g.,  variable 
number  101),  if  available,  and  the  most  commonly  used  hard-card  name  (e.g.,  DCl), 
followed  by  a  short  description  and  the  name  we  assigned  to  the  variable. 

Sample.  The  initial  sample  consisted  of  858  students  who  began  training 
between  August  1990  to  November  1991.  Most  of  the  changes  to  the  curriculum 
occurred  after  Julian  date  91283.  A  preliminary  analysis  found  only  11  cases 
with  data  after  Julian  date  91283.  Also,  157  cases  were  missing  a  registration 
date  and  218  were  missing  complete  data.  Because  of  the  small  number  of  students 
registered  after  91283,  we  conducted  analyses  based  on  students  who  registered 
before  91283.  The  final  sample  consisted  of  483  students  with  complete  data. 

Results  and  Discussion.  Table  62  presents  the  descriptive  statistics  for 
the  set  of  variables  used  in  the  analyses.  We  conducted  a  factor  analysis  on  the 
set  of  knowledge  and  comprehensive  tests.  We  excluded  Digitsl  to  Elecro-Mech.3 
due  to  the  small  number  of  cases  that  had  those  variables.  The  scree  test  and 
the  factor  loadings  indicated  a  two  factor  solution.  Table  63  shows  the 
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Table  6l 


Dcscnmons  of  FC  Variables 


Test  No. 


101 

102 

103 

104 

105 

106 

107 

108 

109 

110 

201 

202 


Description  (Name) 

DCl  (Direct  Current  Oxcuits  1):  Introduction  to  Fire  Control:  Knowledge  of  matter  and  energy;  knowledge*  comprehension, 
and  application  of  metnc  notation  and  electrical  characteristics.  fDC  Circuiul) 

DC2  (Direct  Current  dmiits  2):  Knowledge  and  comprehension  of  batteries  and  electrical  safety:  application  of  scries  DC 
circuits.  (DC  Circutts2) 

DC3  (Direct  Current  Grcuiis  3):  Knowledge,  comprehension,  and  application  of  basic  meters,  multimeters  (Simpson),  and  scries 
circuit  fault  isolation.  (  DC  Circuits3) 

DC4  (Direct  Current  Circuits  4):  Knowledge,  comprehension,  and  application  of  DC  parallel  circuits  and  DC  series-parallel 
circuits.  (DC  Circuiis4) 

DCS  (Direct  Current  Circuits  5):  Knowledge,  comprehension,  and  application  of  DC  series-parallel  circuits  and  voltage  dividers. 
(DC  CircuitsS) 

ACl  (Alternating  Current  Circuits  1):  Knowledge  and  comprehension  of  AC  generation:  knowledge,  comprehension,  and 
application  of  AC  wave  forms  and  AC  test  equipment.  (AC  Circuitsl) 

AC2  (Alternating  Current  Circuits  2):  Knowledge,  comprehension,  and  application  of  inductors,  inductance  circuits,  and 
resist ive/inductive  circuits.  (AC  Circuits2) 

AC3  (Alternating  Current  Circuits  3):  Knowledge,  comprehension,  and  application  of  capacitance,  capacitive  reactance,  series 
resistive/capaciiive  circuits,  and  parallel  resistivc/capacitivc  circuits.  (AC  Circuiis3) 

AC4  (Alternaring  Current  Circuits  4):  Knowledge*  comprehension,  and  application  of  series  rcsisiive/inductivc/capacitive  circuits, 
parallel  resistive/induciivc/capacitive  circuits,  AC  power,  series  resonance*  parallel  resonance,  and  tuning.  (AC  Circuits4) 

ACS  (Alternating  Current  Qicuits  5):  Knowledge*  comprehension,  and  application  of  complex  filters  and  transformers: 
knowledge  and  comprehension  of  circuit  protection  devices  and  circuit  control.  (AC  CircuiisS) 

FCEl  (Tubes);  Knowledge  and  comprehension  of  tubes;  knowledge,  comprehension,  and  application  of  special  tubes  and  tube 
biasing.  (Tubes) 

FCE2  (Transistors);  Knowledge,  comprehension,  and  application  transistors,  field  effect  transistors,  and  special  devices. 
(Transistors) 


Before  class  91283.  the  school  taught  Amplifiers  I  and  II  before  Power  Supplies. 

203  FCE3  (Amplifiers  1):  Knowledge  of  common  emitter  amplifiers*  amplifier  configurations,  and  classes  of  operation;  knowledge, 
comprehension,  and  application  of  linear  integrated  circuits,  operational  amplifiers,  and  special  design  amplifiers.  (Amplifiers!) 

204  FCB4  (Amplificfs  II):  Knowledge,  comprehension,  and  application  of  amplifier  coupling,  amplifier  frequency  range,  and 
amplifier  troubleshooting.  (Amplificrs2) 

205  FCE5  (Power  Supplies>  Knowledge,  comprehension,  and  application  of  basic  power  supplies,  powr  supply  transformers, 
reaifiers,  power  supply  filters,  regulation  and  regulators,  and  introduction  to  solid  state  power  supplies.  (Power  Supplies) 

206  FCE6  (Osdllatois):  Knowledge,  comprehension,  and  application  of  crystal  oscillators,  lesistivc/capacitive  phase  shift  osallators, 
Wein-Bridge  oscillators,  and  blocking  oscillators.  (Oscillators) 

207  FCE7  (Multivibratora):  Knowledge,  comprehension,  and  application  of  bistable  multivibrators,  monostable  multivibrators,  asuble 
multivibrators,  and  Schmitt  Trigger  multivibrators.  (Multivibrators) 
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Table  M  i  C'^'nimued) 


Test  No.  Descnptjon  (Name) 

20S  FCE8  (Spedai  Circuits):  Knowledge,  comprehension,  and  application  of  coincidence  circuits,  sawtooth  circuits,  limiters,  and 
clampers.  (Special  Circuits) 

209  PCE9  (Supcrbcl  1):  Knowledge,  comprehension,  and  application  of  AM  modulators.  FM  modulators,  mixers,  detectors,  and 
discriminators.  (Superhctl) 

2 10  FCEIO  (SoWcr/Dcsokkr):  Knowledge,  comprehension,  and  application  of  solder/desolder  techniques,  high  reliability  soldering, 
uinng.  cable,  and  electro-static  discharge.  (Solder) 

301  DGl  -  Digits  1:  Knowledge  and  comprehension  of  number  s\stems  and  logic  circuits.  (Digitsl) 

302  DG2  -  Digits  Z  Knowledge  of  flip  flop:  knowledge,  comprehension,  and  application  of  digital  counters,  registers/ 
conveners,  and  adders/subtractors.  (Digits2) 

303  DG3  -  Digits  3:  Knowledge,  comprehension,  and  application  of  timers/comparators/pariry:  knowledge,  comprehension,  and 
application  of  l-O/multiplexcrs/cncoders/decoders/storage  devices:  knowledge  of  digital  computers /flow  chaning.  (Digiis3) 

401  EMI  -  Dcctio-McchanicaJ  1:  Knowledge,  comprehension,  and  application  of  AC/DC  generators.  tachomotors/AC  and  DC 
motors.  Amplidyncs/ motor  and  generator  safery/  maintenance:  knowledge  comprehension,  and  application  of  thermal 
proieciion/namcplatc  parameters/noise  and  vibration.  (ElectroMech.l) 

402  EM2  -  Etectro-Mcchanical  Z  Knowledge,  comprehension,  and  application  of  torque  synchros,  troubleshooting  synchros,  torque 
differential  synchros,  and  control  svmchros:  knowledge  and  comprehension  of  scale  factors,  electronic  computing  elements, 
summation  loops,  muliipliers-dividcrs.  and  resolvers.  (Electro-Mech.2) 

403  EM3  -  Ekctro-Mechanical  3:  Knowledge,  comprehension,  and  application  of  servo  mechanisms,  servo  power  systems, 
operation,  gyroscopes,  rate  gyros/acceleromeiers.  and  iniro/operations  of  director  systems.  (Elcctro-Mech.3) 


Prior  to  March  18, 1991.  Radar  (Unit  500)  lasted  5  weeks.  On  this  dale,  the  school  discontinued  troubleshooting  procedures  and  laboratory 
exercises.  This  cut  the  Radar  unit  from  5  weeks  to  four  weeks  in  length.  The  school  never  administered  tests  covering  this  material, 
therefore,  the  tested  course  content  remained  the  same. 

501  R1  (Radar  1):  Knowledge  of  safety/hazards/sccurity.  intro/basics/subsystems.  range/bearing/elcvation,  resolution/accuracy, 
pulsed /pulsed  Doppler/FM-CW  Doppler,  timing,  and  synchronizer  opcraiions/outputs/mainienancc.  (Radarl) 

502  R2  (Radar  2):  Knowledge,  comprehension,  and  application  of  indicators,  indicator  maintenance,  transmission  lines/quaner  wave 
lines,  waveguide/ microwave  components,  klystrons,  traveling  wave  tubes,  and  magnetrons/amplitrons.  (Radar2) 

503  R3  (Radar  3):  Knowledge,  comprehension,  and  application  of  transmitters,  modulation,  transmitter  maintenance, 
antcnnas/scrvos/maintenancc/halfwavc  antennas,  and  duplexers/devices/reflex  klystrons.  (Radar3) 

504  R4  (Radar  4):  Knowledge,  comprehension,  and  application  of  receiver  theory/maintcnance/radar  ranging,  dry  air/liquid  cooling, 
cooling-plumbing/power  distribution.  (Radar4) 
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Table  ()- 


DC  Circuusl 
DC  Circuiis2 
DC  Circuits3 
DC  Circuits^ 

DC  Circuiis5 

AC  Circuusl 

AC  Circuus2 

AC  Circuus3 

AC  Circuus4 

AC  CircuiisS 

Tubes 

Transistors 

.Amplifiers! 

Amp!ificrs2 

Povt’cr  Supplies 

Oscillators 

Mulimbrators 

Special  Circuits 

Sv-  jrncil 

SojJcr 

Digilsl 

Dig:iis2 

Digiis3 

Electro-Mechl 

Eieclro-Mech2 

Elect  ro-Mech3 

Radarl 

Ra(lar2 

Radar3 

Radar4 


848 

839 

839 

830 

819 

806 

801 

790 

777 

778 
764 
772 
752 
734 
736 
713 
703 
693 
690 
688 
104 
104 
104 
104 
104 
104 
667 
667 
670 
672 


85.512 

83.914 

83.371 

85542 

81.391 

87.437 

88.987 

81.648 

83.112 

83.458 

79.282 

78.304 

75.342 

79.402 

83.864 

76.914 

77.418 

83.604 

S8572 

89.012 

91.270 

87.863 

85549 

82.654 

80.457 

83.660 

76.956 

84.682 

81554 

81.732 


8.334 

8.499 

11.005 

12546 

13.397 

8.849 

8.727 

11.286 

12.221 

8.484 

10.190 

11.783 

11.725 

9.873 
8561 
10.430 
11.427 
9532 
9.326 
9.007 
6.970 
8.969 
7.316 
8.806 
8.718 

6.873 
8.772 
9.876 

10.726 
10.278 


4J.773 

-0513 

-1.300 

-1.408 

-1.251 

-1.773 

-1.865 

-0.756 

-1.115 

-0593 

-0.400 

-0532 

-0.453 

-0.403 

-0.499 

-0.792 

-0.538 

-0.621 

-1.159 

-2562 

-1.276 

-1.316 

-0.656 

-0.704 

-0.366 

-0561 

-0.307 

-0.697 

A).602 

-0.663 


0.488 

0.015 

4.970 

2.726 

2.482 

9.845 

10.449 

0.709 

1.915 

0562 

0.196 

0.182 

0.055 

0.138 

0.331 

1518 

0546 

0.106 

3.064 

17,306 

2.230 

2.948 

0.981 

0.690 

-0.166 

0.373 

-0.036 

1.228 

0.115 

1.157 


50.00 

5450 

10.00 

1250 

25.00 

10.00 

6.70 

35.70 
20.80 
45.00 
3250 
25.00 
3750 
40.00 
50.00 
1750 
30.00 
4750 
25.00 
10,00 
65.79 
53.13 
5758 
50.00 
60.00 

63.89 
4550 
28.10 
39.40 

25.90 


100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

100.00 

98.00 

97.83 

97.27 

100.00 

100.00 

100.00 

100.00 


communal i ties,  factor  loadings,  and  factor  score  coefficients.  Based  on  the 
results  of  the  factor  analysis,  we  labeled  the  first  factor  Circuits  and 
Electrical  Devices  (Circuits/Devices)  and  the  second  factor  Radar. 

To  examine  whether  a  Circuits/Devices  versus  a  Radar  distinction  could 
explain  aspects  of  school  performance,  we  computed  a  composite  variable  based  on 
the  average  of  all  the  Circuits/Devices  variables  (DC  Circuitsl  to  Special 
Circuits)  and  a  composite  based  on  the  average  of  all  the  Radar  variables  (Radarl 
to  Radar4).  Additional  composites  included  the  Sural  and  Sum2  composites  which 
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Tabic  03 


Communalities.  Factor  Loadings,  and  Factor  Score  Coefficients  for  FC  Vanables 


Factor  Loadines 

Factor  Score  Coefficients 

Variable 

Communalm- 

Factor  1:  Factor  2: 

Circuits/Dcviccs  Radar 

Factor  i:  Faaor  2: 

Circuits/Dcviccs  Radar 

DC  Circuitsl 

.273 

.490 

.180 

.103 

-.035 

DC  Circuits2 

.265 

.478 

.189 

.096 

-.028 

DC  Circuits3 

.294 

.498 

.211 

.103 

-.022 

DC  Circuits4 

.426 

.653 

.000 

.244 

.186 

DC  Circuiis5 

.308 

J24 

.181 

.117 

-.045 

AC  Circuitsl 

.250 

.447 

.225 

.074 

-.002 

AC  Circuns2 

.282 

J03 

,170 

.107 

-.039 

AC  Circuiis3 

.401 

.544 

.324 

.107 

.008 

AC  Circuiis4 

.281 

.430 

.310 

.047 

.(M2 

AC  Circuitsi 

.346 

.427 

.405 

.041 

.077 

Tubes 

.413 

i23 

.373 

.077 

.051 

Transistors 

.468 

.619 

.291 

.158 

-.033 

AM  PI 

.338 

.434 

.387 

.038 

.080 

.amp: 

.408 

458 

.445 

.040 

.106 

POWRSUP 

.158 

.313 

.244 

.035 

.019 

Oscillators 

.403 

.537 

.339 

.096 

.035 

Multivibrators 

.378 

.520 

.327 

.087 

.021 

Special  Circuits 

.324 

.412 

.393 

.028 

.082 

Superhetl 

.112 

.128 

.310 

-.023 

.075 

Solder 

.053 

.094 

.211 

-.012 

.051 

Radarl 

.353 

.242 

.543 

-.056 

.197 

Radar2 

.239 

.165 

.460 

-.052 

.146 

Radar3 

.334 

.199 

.542 

-.072 

.204 

Radar4 

.434 

.091 

.653 

-.156 

.331 

Note.  n=483 


took  a  simple  average  of  all  the  variables  that  loaded  .30  or  greater  on  Factor 
1  and  Factor  2,  respectively. 

We  also  calculated  a  composite  based  on  the  average  of  Digitsl  to  Digits3 
and  a  composite  based  on  the  average  of  Electrol  to  Electro4.  Correlations  among 
the  Digit  variables  ranged  from  .30  to  .35,  while  correlations  among  the  Electro- 
Mech  variables  ranged  from  .26  to  .46.  We  included  the  Digits  and  Electro-Mech 
composites  in  a  correlation  matrix  in  order  to  assess  their  contribution  to  the 
overall  variance. 
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Because  the  FC  school  computed  the  FSG  using  a  70  as  the  minimum  score, 
we  calculated  our  own  Final  School  Grade  (FSG2)  using  the  initial  test  scores, 
and  listed  both  as  composite  variables.  Table  64  shows  the  descriptive 

statistics  for  the  composite  variables. 

We  then  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  65  presents  the  composite  variable  intercorrelations.  Because 
FSG  and  FSG2  correlate  .99,  this  offers  no  support  for  using  initial  test  scores 

to  compute  FSG. 

Circuit/Devices  and  Suml  correlate  highest  (.94)  with  Factor  1.  followed 
closely  by  FSG  (.85).  Similiarly,  Radar  correlates  highest  (.93)  with  Factor  2, 
followed  by  Sum2  (.87).  These  findings  support  using  the  logical  combinations 
of  tests  to  assess  the  dimensions  reflected  in  the  factor  analysis. 


Tabic  64 

Descriptix^  Statistics  for  FC  Composite  Variables 


Composite 

Rel* 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Suml^ 

Sum2‘^ 

FSG 

FSG2'^ 

Radar® 

Circ./Deviccs^ 

Digits* 

Elec.  Mech.*' 

,895 

.866 

N/A 

.924 

.675 

.893 

.630 

596 

83.0S9 

81.772 

83.094 

84.835 

82.390 

83.450 

88.227 

82.257 

5.890 

5.833 

6.003 

4.906 

7.029 

5.739 

5.753 

6.112 

,130 

.121 

-.497 

.005 

-.342 

.114 

-.775 

-.228 

-.677 

-.196 

1.181 

-.178 

.099 

-.654 

.478 

-.091 

68.18 

65,41 

50.00 

70.87 

55.61 

69.59 

71.03 

66.07 

97.63 

98.02 

97.70 

97.18 

100.00 

97.75 

97.98 

95.83 

«Thesc  are  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^Suml:  Average  of  all  tests  loading  ,30  or  greater  on  Factor  1. 

®Sum2:  Average  of  all  tests  loading  .30  or  greater  on  Factor  2. 

*^FSG2:  Average  of  all  tests  using  initial  test  scores. 

*  Radar  (Average  of  all  Radar  tests. 

^Circ./Dcvices:  Average  of  DC  Circuitl  to  Special  Circuit  tests. 

^Digits:  Average  of  all  Digit  tests, 

'^Elcc.  Mcch.:  Average  of  all  EIcctro-Mech.  tests. 
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to 


Correlations  Among  FC  Composites  and  Factors 


Composite 

Factor! 

Factor2 

Sum!® 

Sum2^ 

FSG 

FSG2' 

Radar^ 

Circ./ 

Devices® 

Digits^ 

Factor2 

.40 

Sum! 

.94 

.67 

Sum2 

.76 

.87 

.92 

FSG 

.85 

.78 

.95 

.95 

FSG2 

.85 

.SO 

.96 

.96 

.99 

Radar 

.44 

.93 

.67 

.83 

.78 

.80 

Circ./Deviccs 

.94 

.68 

.99 

.93 

.96 

.97 

.67 

Digits 

.52 

.61 

.63 

.68 

.72 

.75 

.59 

.63 

Elec.  Mech.® 

ac* _ * _ 

.48 

.o5 

.61 

.68 

.73 

.73 

M 

.61 

.54 

^  .  - - p-  - - - P - -  p - w..  - - -  .. 

^Sum2:  Average  of  all  tests  loading  .30  or  greater  on  Factor  2. 
';FSG2:  Average  of  all  tests  using  initial  test  scores. 

"Radar  Average  of  all  Radar  tests. 

■Circ./De\ices:  Average  of  DC  Circuit!  to  Special  Circuit  tests. 
•Digits:  Average  of  all  Digit  tests. 

-I  Mech.:  Average  of  all  Elcctro-Mcch.  tests. 


Finally,  FSG  correlates  .96  with  Circuits/Devices  and  .78  with  Radar. 
These  findings  support  using  FSG  as  an  indicator  for  performance  on 
Circuits/Devices  tests  but  suggest  using  a  separate  indicator  for  performance  on 
Radar  tests. 

Recommendations.  In  addition  to  FSG,  use  an  average  of  all  Radar  tests. 
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Gunner’s  Mate  -  Gun  (GMG) 

Description  of  Variables.  The  GMG  school  generates  all  written  tests  with 
the  COGENT  (Computer  Generated  Testing  System)  microcomputer  program.  The  COGENT 
program  makes  each  test  unique  by  randomly  selecting  multiple  choice  questions 
from  a  content  area  test  bank.  The  school  predetermines  the  material  covered  and 
the  number  of  questions  required  for  each  test.  The  GMG  school  currently 
generates  test  scores  through  ISS  and  has  since  June  17,  1991.  We  received  only 
data  for  students  with  a  convening  date  before  June  17,  1991,  and  therefore  the 
data  was  all  hard  card  data. 

The  school  scores  all  tests  on  a  scale  of  0.00  to  100.00  and  requires  a 
minimum  passing  score  of  75.  Students  scoring  below  75  have  the  option  to  take 
a  retest.  Those  choosing  not  to  take  a  retest  keep  their  initial  score.  For 
students  who  choose  to  take  a  retest  and  pass,  the  school  assigns  a  minimum 
passing  score  of  75.  When  a  student  fails  an  individual  test  twice,  the  student 
must  go  before  the  Academic  Review  Board.  The  hard  card  data  we  received  did  not 
include  initial  test  scores.  Therefore,  we  entered  the  assigned  score  of  75  when 
the  student  passed  a  retest. 

Table  66  provides  a  description  of  the  GMG  school  tests  for  both  hard  card 
and  ISS  tape  data.  For  hard  card  data,  the  school  calculated  the  Final  School 
Grade  (FSG)  by  dividing  the  sum  of  the  test  scores  by  the  number  of  tests  given. 
For  ISS  tape  data,  the  school  now  uses  the  following  weighted  average:  Tests  1- 
10  each  constitute  2.9%  of  the  FSG;  Tests  11-25  each  constitute  3.2%  of  the  FSG; 
and  Test  27  constitutes  20%  of  the  FSG. 
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l  ablc  60 


Descnnions  of  GMG  \*anables 


Test  No.  Descnption  (Namc» 


All  tests  are  written  tests 

I  Mathcmaucs  (Math) 

:  Matter  and  Encrg>*,  Electromotive  Force  and  Energv'.  Resistance  and  Resistors.  Electrical  Safety.  Schematics,  and  Multimeter 

Usage  (Energ\'j 

3  Scries  DC  Circuits  (Scries  DC) 

4  Parallel  DC  Circuits  (Parallel  DC) 

5  Magnetism.  AC  Generation.  Wave  Form  Analysis,  and  Oscilloscopes  (Magnetism) 

6  Inductors  and  Inductance.  RL  Circuits,  and  Transformers  (  Inductors) 

7  Capacitors  and  Capacitance.  RC  Circuits,  and  RLC  Circuits  (Capacitors) 

(Beginning  July  30.  1991.  the  school  split  content  areas  from  Tests  8  and  9  to  make  up  Test  10.  and  designated  Test  111  as  the 
new  comprehensive  exam.  The  school  did  not  change  the  content  areas  with  these  changes.) 

8  Electrical  Connections.  Semiconductors.  General  Transistors,  and  Transistor  Configurations  (through  July  30.  1991). 
(Transis.) 

9  Rectifier  Circuits,  Filter  Circuits.  Special  Derices.  Integrated  Circuits  (through  July  30.  1991).  (Rectifiers): 

10  Comprehensive  test  over  maienais  covered  on  tests  1-9  (through  July  30.  1991),  (Comp  1) 

II  Digits  1 

12  Digits  2 

13  Motors  and  Generators  (Motor /Gen) 

14  Special  Circuits  (Sp.  Circ) 

15  Hydraulics  1  (Hydr.  1) 

16  Hydraulics  2  (Hydr.  2) 

17  Scrvos/Synchros  Systems  (Serv/Syn) 

18  Missile  Systems  (Missiles) 

19  Ammunition  (Ammo.) 

20  Magazines  and  Sprinklers  (Mag/Sprink) 

21  Gun  Mounts  (Gun  Mts) 

22  Tag-Out 

23  3M  Maintenance  (3.M) 

24  Handtools 

25  Small  Arms  (S.  Arms  ) 

27/30  Comprehensive  test  over  material  covered  on  tests  10-25.  (Test  27  corresponds  to  ISS  tape  data,  and  Test  30  corresponds  to 
hard  card  data.  Although  the  tests  have  different  numbers,  the  material  covered  is  identical.)  (Comp  2) 


Sample.  The  sample  consisted  of  301  students  who  began  the  GMG  curriculum 
before  June  17,  1991.  We  excluded  from  some  analyses  the  55  cases  that  did  not 
take  every  test. 

Results  and  Discussion.  We  computed  descriptive  statistics  for  each  of  the 
test  variables.  Table  67  lists  the  descriptive  statistics  for  all  variables. 


102 


Table  tj 


PesrnrTive  Statistics  for  GMG  ^'ariables_ 


Variable  n  Mean  Std  Dev 


Skew  Kurtosis  Minimum  Maximum 


Math 

Encrg>' 

Scries  DC 
Parallel  DC 
Magnetism 
Inductors 
Capacitors 
Transis. 
Rectifiers 
Comp,  i 
Digits  1 
Digits  2 
Motor/ Gen. 
Spec.  Circ. 
Hydr.  1 
Hydr.  2 
Serv/Svm. 
MjS‘;:!cs 
An*  ...o. 
Mag/Spnnk. 
Gun  Mts. 
Tag-out 
3M 

Handtools 
S.  Arms 
Comp.  2 


301 

88.768 

301 

83.610 

301 

88.686 

301 

81i33 

301 

88.245 

301 

89.175 

301 

83.479 

301 

83.388 

298 

82.886 

298 

"9.443 

2S6 

90.724 

277 

83.974 

278 

86.322 

274 

86.253 

268 

84.849 

259 

88.845 

259 

S5.874 

259 

94.079 

259 

88.987 

254 

92.854 

254 

94.728 

253 

90.909 

249 

87.494 

249 

93il6 

249 

85.133 

249 

68.550 

8.036 

-1.505 

5.051 

38.46 

100.00 

8.561 

-.180 

-  .197 

51.15 

100.00 

8.426 

-  .749 

.006 

59.38 

100.00 

10.287 

-  .689 

1.266 

3730 

100.00 

7.800 

-  .740 

.304 

6230 

100.00 

8.088 

-  .640 

-  .187 

6230 

100.00 

8.528 

-  .172 

-.312 

53.13 

100.00 

9.356 

-  .613 

.438 

43.18 

100.00 

8.913 

-  i58 

.425 

50.00 

100.00 

8.353 

-  .086 

-.317 

5536 

100.00 

8-581 

-1.060 

.668 

5536 

100.00 

11.742 

-.948 

.922 

39.29 

100.00 

7.974 

-  .618 

.484 

59.38 

100.00 

8.124 

-  .789 

1.635 

44.44 

100.00 

9.448 

-  .778 

.651 

50.00 

100,00 

7.565 

-  .789 

.951 

55.26 

100.00 

9.377 

-  .902 

1.051 

50.00 

100.00 

5.521 

-1.061 

.892 

75.00 

100.00 

8.101 

-  .750 

.357 

61.36 

100.00 

6.279 

-1.083 

1.162 

66.67 

100.00 

5.775 

-1.302 

1.220 

75.00 

100.00 

7.907 

-1.158 

2.371 

50.00 

100.00 

9.436 

-  .834 

.719 

52.78 

100.00 

6.041 

-1.392 

1.819 

70.83 

100.00 

8.850 

-385 

.442 

50.00 

100.00 

9.173 

.351 

.157 

43.00 

100.00 

Next,  we  conducted  a  factor  analysis  on  the  set  of  all  test  variables.  The 
scree  test  and  the  factor  loadinQS  indicated  two  factors.  Based  on  the  results 
of  the  factor  analysis,  we  labeled  the  first  factor  1st  Half  and  the  second 
factor  2nd  Half.  Tests  1  to  13  (up  through  Motor/Gen.  in  Table  66)  loaded  higher 
on  Factor  1  and  tests  15  to  27/30  loaded  higher  on  Factor  2.  Test  14  (Spec. 
Circ.)  loaded  about  evenly  on  both  factors.  Table  68  lists  the  communal i ties, 
factor  loadings,  and  factor  score  coefficients  for  each  of  the  variables. 
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Table  bS 


Communaiuies.  Facior  i  oaciini:?.  and  Factor  Score  Coefficiems  for  GMG  Vanahles 


Variable 

Communality 

Factor  Loadings 

Factor  Score  Coefficients 

Factor  1: 

1st  Half 

Factor  2: 

2nd  Half 

Factor  1: 

1st  Half 

Factor  2: 

2nd  Half 

Math 

.209 

.437 

.139 

.086 

-.034 

Encrg>’ 

.380 

591 

.155 

.169 

-.073 

Series  DC 

.393 

.608 

.151 

.180 

-.087 

Parallel  DC 

.408 

.622 

.147 

.192 

-.091 

Magnetism 

.273 

.468 

.2>4 

.094 

-.021 

Inductors 

.217 

.357 

.299 

.WO 

.019 

Capacitors 

.361 

i56 

.227 

.135 

-.033 

Transis 

.416 

il6 

.386 

.098 

-.017 

Rectifiers 

.366 

.471 

.380 

,082 

.032 

Comp,  1 

.441 

.606 

.271 

.163 

-.W3 

Dieits  1 

.3^07 

.509 

.220 

.112 

-.031 

Digits  2 

326 

.502 

.270 

.106 

-.014 

Motor/ Gen 

.320 

.424 

.374 

.052 

.039 

Spec.  Circ. 

.248 

.333 

.370 

.029 

.W1 

Hvdr.  1 

.375 

.348 

.503 

.010 

.096 

Hydr.  2 

.323 

.270 

-500 

-.014 

.100 

Serv/Svm. 

.286 

.347 

.406 

.031 

.054 

Missiles 

.270 

.202 

.478 

-.029 

.099 

Ammo. 

.396 

.193 

.599 

-.066 

.169 

Mag/Sprink. 

.345 

.156 

561 

-.060 

.146 

Gun  Mts. 

.307 

.221 

509 

-.030 

.110 

Tag-out 

.442 

.039 

.664 

-.150 

.250 

3M 

.346 

.312 

.499 

.001 

.099 

Handtools 

.347 

.226 

544 

-.035 

.129 

S.  Arms 

.324 

.257 

-508 

-.024 

.102 

Comp.2 

.420 

.359 

-539 

.003 

.122 

Note,  n  *  246 

In  order  to  determine  whether  the  separation 

of  the  course  curriculum  could 

explain  aspects  of  student  performance,  we  created 

a  1st  Half  composite  variable, 

which  was 

an  average  of  tests 

1  to 

14,  and  a  2nd 

Half  composite 

variable  which 

was  an  average  of  tests  14  to  27/30. 

We  included 

test  14  (Spec. 

Circ.)  in  both 

composites 

because  it  loaded 

about  equally  high 

on  both  factors.  The  school 

gives  a  comprehensive  test 

over 

the  material 

covered  by  tests  1-9  and  a 

comprehens 

ive  test  over  the  material  covered  by 

tests  11-25. 

We  created  two 

composite 

variables,  Phasel 

and 

Phase2,  based  on  this  separation  also. 

Additional 

composites  included  the  Suml  and  Sum2 

composite  variables  which  took 
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a  simple  average  of  all  tests  that  loaded  .30  or  greater  on  either  Factor  1  or 
Factor  2.  respectively.  We  included  tests  that  loaded  above  .30  on  both  Factor 
1  and  Factor  2  in  the  averages  for  both  composite  variables.  We  also  added  the 
student’s  FSG  as  a  composite  variable.  We  did  not  compute  our  own  FSG  because 
initial  test  scores  were  not  available.  We  used  the  factor  scores,  Factor  1  and 
Factor  2,  as  additional  composite  variables.  Table  69  lists  the  descriptive 
statistics  for  each  of  the  composite  variables  listed  above.  The  n  varied  for 
the  composite  variables  depending  on  the  number  of  tests  missing  for  each  case. 


Table  69 


Descnpfive  Statistics  For  GMG  Composite  V^ariables 


Composite 

n 

Rel* 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Suml^ 

247 

.89 

84.812 

5.273 

.000 

-.533 

73.17 

9730 

Sum2‘= 

247 

.89 

87.247 

4.860 

-.081 

-.474 

72.89 

97.81 

FSG 

301 

.92 

86.340 

4.893 

-.030 

-339 

73.15 

98.02 

Halfl° 

273 

.87 

85372 

5.266 

.016 

-.635 

74.23 

97.26 

Half2® 

247 

.87 

87.890 

4.835 

-.178 

-.398 

72.21 

98.25 

Phase  1^ 

298 

.84 

84.918 

5.455 

-.013 

-310 

72.25 

97.93 

Phasc2® 

246 

.88 

87.775 

4.848 

-.106 

-390 

74.04 

98.38 

aThese  arc  composites  based  on  communaliw  estimates  and  the  procedures  described  in  the  text. 
'^Suml:  Average  of  Tests  with  Factor  1  loadings  greater  than  .30. 

‘^Sum2:  Average  of  Tests  with  Factor  2  loadings  greater  than  30. 

FSG:  Student's  FSG  using  retest  scores. 

^Halfl:  Average  of  Tests  1-14. 

®Half2:  Average  of  Tests  14-27/30. 

^Phascl:  Average  of  Tests  1-10. 

®Phase2:  Average  of  Tests  11-27/30. 
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'rablc  7'.' 


Correiations  Among  GMG  Composites  and  Factors 


Comp 

I  aciorl 

Facior2 

Suml^ 

Sum2*^ 

FSG 

Halfr 

Half2'=‘ 

Phase!®  Phasc2^ 

Factorl 

- 

Factor2 

.n 

Suml 

ss 

.59 

- 

Sum2 

.56 

.90 

.87 

- 

FSG 

-8 

.74 

.98 

.94 

- 

Halfl 

.93 

.48 

.98 

.79 

.93 

- 

Half2 

.48 

.93 

.82 

.98 

.91 

.72 

- 

Phase! 

.93 

.42 

.93 

.74 

.90 

.96 

.66 

- 

Phasc2 

.55 

.87 

.88 

.98 

.95 

.80 

.97 

.71 

®Suml:  Average  of  Tests  with  Factor  1  loadings  greater  than  .30. 

^Sum2:  Average  of  Tests  with  Factor  2  loadings  greater  than  .30. 

FSG:  Student  s  FSG  using  retest  scores. 

^Halfl:  Average  of  Tests  1-14. 

"Ha!f2:  Average  of  Tests  14-27. 

^Phasci:  Average  of  Tests  1-10. 

“Phase2;  Average  of  Tests  11-27/30. 


Finally,  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  70  provides  the  intercorrelations  between  all  composite 
variables.  Halfl  and  Phasel  both  correlate  .93  with  Factorl,  followed  by  Suml 
(.88).  Half2  correlated  highest  with  Factor2  (.93),  followed  by  Sum2  (.90)  and 
Phase2  (.87).  This  supports  using  the  logical  combination  of  variables  to 
measure  the  factors  observed.  Finally,  FSG  correlates  .78  and  .74,  respectively, 
with  Factorl  and  Factor2  but  .93  and  .94  with  Halfl  and  Half2. 

Recommendations.  In  addition  to  FSG,  use  an  average  of  all  Halfl  tests  and 
average  of  all  Half2  tests. 
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Machinist’s  Mate  (MMl) 

Before  September  27,  1990,  all  MM  students  took  Propulsion  Engineering 
Basics  (PE)  as  a  prerequisite  course.  Instruction  for  the  PE  and  the  MM  courses 
took  place  in  the  same  building  of  the  Great  Lakes,  IL  Service  School  Command. 
Civilian  personnel  taught  PE,  and  military  personnel  taught  MM.  As  of  September 
27,  1990,  the  MM  school  incorporated  the  PE  course  into  the  MM  course  program. 
As  a  result  of  this  merger,  teaching  responsibilities  for  the  PE  course  shifted 
from  civilian  to  military  personnel.  However,  the  PE  course  content  and  its 
presentation  schedule  did  not  change  at  this  time. 

On  January  7,  1991,  the  school  revised  the  MM  curriculum  to  further 
integrate  the  PE  and  MM  course  material.  The  present  section  covers  students 
entering  the  MM  school  before  January  7,  1991,  when  the  PE  and  the  MM  course 
material  remained  distinct.  We  labeled  this  curriculum  MMl.  The  subsequent 
curriculum,  MM2,  addresses  students  entering  the  MM  school  after  the  instructors 
integrated  the  PE  and  the  MM  course  material  (January  7,  1991). 

Description  of  Variables.  Table  71  presents  the  MMl  tests  and  their 
corresponding  descriptions.  The  MM  school  assesses  student  progress  using 
written  knowledge  and  comprehensive  tests.  The  tests  have  a  valid  range  of  0  to 
100.  Within  the  MMl  curriculum,  the  first  six  tests  cover  PE  material.  The 
remaining  eight  tests  address  MM  issues.  The  school  requires  a  score  of  75%  or 
higher  to  pass  each  exam.  Examinees  who  fail  a  test  participate  in  instructor 
oral  remediation,  night  study,  and  retesting.  Students  cannot  retest  until  the 
next  academic  day.  All  students  who  fail  the  retest  appear  before  the  Academic 
Review  Board.  The  school  then  decides  upon  the  student’s  status.  The  instructor 
assigns  the  minimum  passing  score  to  those  students  who  pass  the  retest,  and  the 
school  uses  that  minimum  score  in  calculating  the  final  school  grade  (FSG). 
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Table  71 


Descnptions  of  MMl  Variables 


Var,  No.  Description  (Name) 


105  Knowledge  Test  Personnel  qualification  standards/communication  depanmeni  watch  organization,  introduction  to 

shipboard  piping  systems,  piping  system  components,  tag-out  (Piping  System) 

109  Knowledge  Test  Firemam/auxiiiary  cooling  water,  main  drain  potable  &  waste  water  system,  twin  agent  system,  halon 

fire  system  (^ater  S>stem) 

114  Knowledge  Test  Maintenance  &  material  management  system  (S-M),  equipment  technical  manuals,  hand  tools/metal 

fastners.  lubricanis/packing/gaskeis/insulation  (5-M  System) 

117  Knowledge  Test  Valves,  couplings/gears/bearings,  roiary/jet/centrifugal/rcciprocating  pumps  (Valves) 

122  Knowledge  Test  Lubrication  system,  lube  oil  quality  program/fill/transfer/purification.  introduction  to  cold  iron 

(Lube  S>stem) 

125  Coenprebensive  Test  1:  Introduction  to  piping  systems,  piping  systems,  intro  to  maintnance.  maintenance,  lube  oil 

system,  cold  iron  (PE  Comp) 

100  Knowledge  Test  Basic  steam  cycle,  energy  curve,  flue  gas/watcr/steam  path,  fuel  oil  system  (Steam  Cycle) 

200  Knowledge  Test  Basic  engineering  terms,  combustion  air,  superheater  protection  steam/main  steam  auxiliary/reduced 

pressure  (Engineenng) 

300  KncMrlcdge  Test  Turbine  theory  terms,  turbines,  mail  reduction  gear/shafting/lube  oil,  purification  and  transfer,  lube 

oil  quality  management  (Turbine  Theory) 

4(XI  Knowledge  Test  Gland  seal/exhaust  purpose,  gland  seal  &  assembly,  regulating  station,  main  condenser/seawater 

circulation  theory,  propeller  pump  (Gland  Seal) 

500  Knowledge  Test  Pump  theory  terms,  main  air  removal /condensate/feed,  deaerating  feed  tank,  freshwater  collecting 

tank,  high  pressure  drain  system,  auxiliary  exhaust  system  (Pump  Theory) 

6(XI  Knowledge  Test  Distilling  plant,  excess /make*up/reserve  feed,  escape  steam  system,  physical/chemical  propenies 

of  water,  corrosion  factors,  shipboard  water  cycle,  condensate/feedwater  analysis,  boiler  lay-ups  (Steam  System) 

700  Knowledge  Test  Ship  sen'ice  turbo  generator  components,  basic  electricity  theories  &  circuit,  A.C  generator,  ships 

service  power  distribution  system,  sources  of  electrical  energy,  alarm  system  (Generator) 

800  Comprtbenaw  Test  2;  Generation,  steam  systems,  turbine  theory/drive  train,  gland  seal/exhaust,  main  condenser/ 

condensate/feed,  distilling  plant/water  chemistry,  electricity  (MM  Comp) 


Note.  Tha  school  computas  a  PE  grade  and  an  grade  as: 

P£  Grade  *  .8  (naan  of  Piping  System,  Water  System,  3'M  System,  Valves,  and  Lube  System)  +  .2(FE  Comp). 
Hi  Grade  «  .8  (mean  of  Steam  Cycle,  Engineering,  Turbine  theory,  Gland  Seal,  Pump  Theory,  Steam  System, 
and  Generator)  +  .2(tf1  Comp). 

The  school  computes  FSG  as  the  mean  of  the  PE  Grade  and  the  Grade. 


108 


Although  the  MM  school  assigns  a  score  of  75  for  all  successful  retests,  for  the 
purpose  of  the  present  study,  RGI  entered  all  initial  test  scores  into  the 
database. 

For  each  student,  the  school  computes  a  PE  grade  and  an  MM  grade  from  a 
weighted  average  of  the  corresponding  knowledge  and  comprehensive  test  scores. 
The  school  uses  the  mean  of  the  PE  grade  and  the  MM  grade  as  the  FSG.  See  Table 
71  for  tests  weights. 

Sample.  The  initial  sample  contained  373  students  who  began  training 
between  April  1990  and  January  1991.  The  school  merged  the  PE  course  with  the 
MM  course  while  the  MMl  curricul urn  was  in  effect.  Civilian  instructors  taught  PE 
tr  the  earlier  students  of  the  MMl  format  (n=66)  and  military  personnel  taught 
PE  to  students  who  took  the  course  later  (n=307). 

Results  and  Discussion.  We  began  by  testing  the  two  groups  of  students  for' 
differences  due  to  the  change  in  PE  instructors.  We  conducted  a  multivariate 
analysis  of  variance  (MANOVA)  to  test  the  correlation  matrices  of  the  test 
scores.  A  test  for  homogeneity  of  covariance  matrices  revealed  that  the  two 
forms  had  significantly  different  covariance  matrices  (F=1.67,  £<.001).  The  test 
means  also  differed  significantly  across  the  two  forms  (F=2.63,  £<.001). 
Univariate  F-tests  indicated  that  all  variables  differed  between  forms  at  the 
£<.05  level  (except  the  following  tests:  3-M  System,  Lube  System,  and  Steam 
System).  These  findings  indicate  that  the  two  samples  come  from  two  distinct 
populations. 

To  test  whether  ability  accounted  for  these  differences  between  the  two 
groups  of  students,  RGI  conducted  a  multivariate  analysis  of  covariance  (MANCOVA) 
using  the  students’  pre-enlistment  ASVAB  scores  as  covariates.  After  parti  ailing 
out  the  variance  due  to  ASVAB  scores,  test  for  homogeneity  of  covariance  matrices 
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revealed  different  matrices  for  the  two  forms  (F=1.19,  £<.05).  Test  means  also 
differed  across  the  two  forms  after  controlling  for  ASVAB  scores  (F=2.56,  £<.01). 
These  results  suggest  that  the  noted  difference  between  the  two  groups  is  due  to 
something  other  than  differences  in  ability.  Based  on  these  findings,  RGI 
analyzed  each  group  separately. 

To  further  evaluate  the  conclusion  of  differences  between  groups,  we 
computed  effect  sizes  (ES)  as  measures  of  the  practical  significance  of  the 
differences  between  the  unadjusted  means  across  groups.  Cohen  and  Cohen  (1977) 
state  that  .2  represents  a  small  effect  size,  .5  is  medium,  and  .8  is  large.  For 
a  total  of  14  comparisons  between  the  MM  groups,  we  found  1  ES  within  the  .00  to 
.19  range,  11  ESs  within  the  .20  to  .49  range,  and  2  ESs  within  the  .50  to  .79 
range.  These  results  show  that  85%  of  the  ESs  fall  below  a  value  of  .50.  Thus, 
while  the  MANOVA  tests  indicate  that  the  groups  differ  from  one  another  in  terms 
of  statistical  significance,  the  ESs  suggest  small  magnitudes  of  the  differences, 
in  terms  of  practical  significance. 

Within  the  MMl  curriculum.  Form  1  contains  students  who  took  the  PE  course 
separately,  under  civilian  instruction  (n=66).  Form  2  contains  students  taking 
the  PE  course  as  the  first  part  of  the  MM  course,  under  military  instruction 
(n=307).  Form  1  included  two  cases  with  missing  data.  RGI  excluded  these  cases 
from  some  of  the  analyses,  resulting  in  a  final  sample  size  of  64  cases  for  Form 

1.  Form  2  included  23  cases  with  missing  data.  Likewise,  we  excluded  these 
cases  from  some  analyses,  resulting  in  a  final  sample  size  of  284  cases  for  Form 

2. 

RGI  computed  descriptive  statistics  for  each  of  the  test  variables.  Table 
72  presents  these  statistics,  listing  them  separately  for  Form  1  and  Form  2. 
Next,  we  conducted  a  factor  analysis  on  each  form  separately.  We  included  the 
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Tabic  72 


Descnrtive  Statistics  for  MMl  Variables 


V'ariabie 

ji 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Form  1  _ 

Piping  Sj'stcm 

66 

88.182 

10.227 

-1.285 

1.022 

56.CO 

100.00 

Water  S>’sicm 

66 

88.061 

10519 

-1.638 

2.282 

56.00 

100.00 

3-M  Sv^tem 

66 

83.970 

9.713 

-  544 

-  524 

62.00 

100.00 

Valves 

66 

84.818 

11.762 

-  .611 

-  .367 

54.00 

100.00 

Lube  S>-stcm 

66 

86545 

12.461 

-2.823 

13.243 

18.00 

100.00 

PE  Comp 

66 

86.621 

7.007 

-.283 

-  .059 

69.00 

100.00 

Steam  Cycle 

66 

92.424 

6.888 

-1.460 

3-202 

66.00 

100,00 

Enginecnng 

66 

87.879 

9.073 

-  .438 

-  .775 

68.00 

100.00 

Turbine  Thcor>’ 

66 

90.697 

9.776 

-3.350 

16.816 

34.00 

100.00 

Gland  Seal 

64 

91.312 

7527 

-1.298 

1.955 

68,00 

100.00 

Pump  Theory’ 

64 

90.187 

8.325 

-  .779 

-  .366 

70.00 

100.00 

Steam  S>*stcm 

66 

87.469 

12.923 

-3.191 

15.778 

14.00 

100.00 

Generator 

64 

90.781 

7.238 

-1.017 

1.373 

66.00 

100.00 

MM  Comp 

64 

85.750 

10.234 

-2.916 

14.118 

29.00 

100.00 

FSG 

66 

89.997 

5.215 

-.497 

-  .313 

77.01 

98.40 

Form  2 

Piping  System 

306 

84.072 

9.463 

-  .842 

.903 

44.00 

100.00 

Water  System 

303 

84.264 

10.241 

-.669 

-  .119 

46.00 

100.00 

3-M  System 

297 

82545 

9.458 

-  .894 

1.647 

40.00 

100.00 

Valves 

294 

81.109 

11565 

-  .606 

.250 

42.00 

100.00 

Lube  System 

293 

83.843 

9.102 

-  .673 

507 

50.00 

100.00 

PE  Comp 

291 

84.048 

7.211 

-  .375 

.126 

59.00 

99.00 

Steam  Cycle 

288 

89.354 

8.132 

-1.064 

1.017 

56.00 

100.00 

Engineering 

287 

84.962 

9.050 

-  .911 

.939 

52.00 

100.00 

Turbine  Theory 

285 

86.982 

7.663 

-  .774 

.180 

66.00 

100.00 

Gland  Seal 

284 

87.993 

7.610 

-  .848 

.407 

62.00 

100.00 

Pump  Theory 

284 

86.218 

8.977 

-  .783 

.693 

54.00 

100.00 

Steam  System 

284 

85.063 

9.688 

-1.776 

7538 

26.00 

100.00 

Generator 

284 

85.944 

7.807 

-  ,481 

-  .148 

64.00 

100.00 

MM  Comp 

284 

82.655 

7.002 

-  .276 

-  .349 

64.00 

98.00 

FSG 

306 

85.007 

5.350 

-545 

2.276 

6050 

9750 

set  of  all  test  variables  in  these  analyses.  Resulting  scree  tests  and  factor 
loadings  for  the  two  forms  were  similar,  both  indicating  a  one-factor  solution. 

Table  73  shows  the  communal i ties,  factor  loadings,  and  factor  score 
coefficients  for  Form  1  and  Form  2.  For  both  forms,  all  of  the  tests  loaded 
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Tabic  73 


Communalmcs.  Factor  Loadings  and  Factor  Score  Coefficients  for  MMl  \*ariables 


Communaiity 

Factor  Loadings 

Factor  Score  Coefficients 

Variable 

Form  1  Form  2 

Form  1  Form  2 

Form  1  Form  2 

Piping  System 

.414 

.365 

.644 

.604 

.072 

.118 

Water  Svstem 

.523 

.361 

.723 

.601 

.144 

.106 

S-M  Sy-sicm 

.397 

.239 

.630 

.489 

.076 

.080 

Valves 

.525 

.469 

.725 

.685 

.071 

.149 

Lube  System 

.211 

.313 

.460 

J59 

.015 

.094 

PE  Comp 

.690 

.349 

.831 

591 

.255 

.104 

Steam  Cycle 

.322 

.332 

.568 

576 

.081 

.101 

Engineering 

.628 

.405 

.792 

.636 

.223 

.133 

Turbine  Theory' 

.370 

.275 

.609 

524 

.067 

.083 

Gland  Seal 

.405 

.343 

.636 

586 

.079 

.104 

Pump  Theory 

.417 

.297 

.646 

545 

.057 

.093 

Steam  System 

.149 

.170 

.385 

.412 

.033 

.068 

C  -.eraior 

.310 

.243 

SSI 

.493 

.073 

,078 

ViM  Comp 

.378 

-511 

.614 

.715 

.082 

.176 

Note.  For  Form  1,  ji  =  64.  For  Form  2.n  -  284. 


highly  on  the  factor.  Factor  loadings  ranged  from  .385  to  .831  for  Form  1  and 
.412  to  .715  for  Form  2. 

We  sought  to  develop  composite  variables  which  would  capture  essential 
elements  of  the  criterion  variance  and  also  represent  meaningful  content 
dimensions  useful  to  the  MM  school.  In  order  to  examine  whether  a  PE  section 
versus  an  MM  section  distinction  could  explain  aspects  of  school  performance,  we 
developed  a  PE  Grade  composite  and  an  MM  Grade  composite  (see  Table  1  for 
computations  of  these  composites).  Since  the  MM  school  computed  the  FSG  using 
a  75  as  the  minimum  score,  RGI  calculated  its  own  final  grade  (FSG2)  using 

initial  test  scores  and  listed  both  as  composite  variables.  We  used  the  factor 

% 

score  (Factorl)  as  an  additional  composite.  Table  74  shows  the  descriptive 
statistics  for  each  of  the  composite  variables  mentioned  above.  We  placed  all 
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Table  74 


Desenrtive  Statistics  for  MMl  Comrvosite  \  anahies 


Composite 

n 

Rel* 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Form  1 

FSG 

66 

89.997 

5.215 

-.497 

-.313 

77.10 

98.40 

FSG2° 

64 

.903 

87.810 

6.288 

-.485 

-  .439 

74.23 

97.10 

PE  Grade'= 

66 

.SAS 

86.376 

6.288 

-.485 

-  .439 

74.23 

97.10 

MM  Graded 

64 

.821 

89.350 

5.983 

-.710 

-  .060 

75.14 

98.49 

Form  2 

FSG 

306 

85.007 

5.350 

-345 

2.276 

60.50 

97J0 

FSG2 

284 

.sas 

84.749 

5.373 

.181 

-  .669 

69.91 

97.39 

PE  Grade 

291 

.819 

83-522 

6.379 

.054 

-  .338 

63.96 

97.68 

MM  Grade 

2&4 

.507 

85.878 

5.262 

-.069 

-  .634 

71.71 

9734 

Note.  For 

Form  1,  I* 

*  6^.  For  Form  2, 

n  ■  284. 

®  These  are  composites  based  on  comnunality  estimates  and  the  procedures  described  in  the  text. 

^  FSG2:  (PE  Grade  -*■  hW  Grade )/2. 

^  PE  Grade:  .5  (mean  of  Piping  System.  Water  System.  3-M  System,  Valves,  and  Lube  System)  +  .2(PE  Comp). 

^  hW  Grade:  .8  (mean  of  Steam  Cycle.  Engineering,  Turbine  Theory,  Gland  Seal,  Pump  Theory,  Steam  System,  and 
m  Generator)  .2 (Mi  Comp). 


the  composite  variables  on  a  scale  of  0-100  in  order  to  compare  them  more 
readily.  The  sample  size  varied  for  the  composites  depending  on  the  number  of 
tests  missing  in  each  case. 

Finally,  we  computed  pearson  correlation  coefficients  for  the  composite 
variables.  Table  75  presents  these  correlations.  For  both  Form  1  and  Form  2, 
the  PE  Grade  and  MM  Grade  scores  correlated  greater  than  .90  with  the.  Factor! 
composite  variable.  Also  noted  on  Form  1  and  Form  2,  both  final  school  grades 
(FSG  and  FG)  correlated  strongly  with  Factor  1.  Although  the  MANOVA  and  MANCOVA 
tests  revealed  Form  1  and  Form  2  to  be  significantly  different  groups,  they 
demonstrated  similar  patterns  of  relationships  with  the  other  composites.  To 
determine  whether  the  two  forms  have  the  same  underlying  factor,  we  computed  the 
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Tabic  75 


Correlations  Among  MMl  Comnosites  and  Factors 


Composite 

Factorl 

FSG 

FSG2“ 

PE  Graded’ 

MM  Grade® 

Factorl 

.98 

.99 

.91 

.93 

FSG 

.93 

— 

,98 

.91 

.90 

FSG2 

.98 

.92 

— 

.94 

.91 

PE  Grade 

.95 

.78 

.95 

— 

.70 

MM  Grade 

.91 

.94 

.91 

.73 

Sote .  Coefficients  below  the  diagonal  correspond  to  Form  1  (n  «  64).  Coefficients  above  the  diagonal 
correspond  to  Form  2  (n  *  204). 

*  FG;  (?E  Grade  +  m  Grade )/2. 

^  PE  Grade:  ,8  (mean  of  Piping  System,  Water  System.  3-M  System,  Valves,  and  Lube  System)  +  .2 (PE  Comp). 

^  Grade:  ,3  (mean  of  Steam  Cycle.  Engineering,  Turbine  Theory,  Gland  Seal,  Pump  Theory,  Steam  System,  and 
m  Generator)  +  .2(^t1  Comp). 


congruence  coefficient  between  the  factor  loadings  from  each  form  (Horsuch,  1983, 
p.  285,  Congruence  =  .99).  This  indicates  that  the  factor  patterns  of  the  two 
forms  are  similar.  However,  the  small  sample  size  of  Form  1  (n=64),  suggests 
that  the  results  of  the  Form  1  factor  analysis  may  have  low  stability. 

Recommendations.  RGI  recommends  using  the  school -calculated  FSG. 
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Machinist’s  Mate  (MM2) 

On  January  7,  1991,  the  MM  school  integrated  the  Propulsion  Engineering 
Basics  (PE)  course  material  into  its  curriculum.  At  this  time,  the  school 
changed  the  course  curriculum  and  the  test  content  to  avoid  redundancy.  The 
preceding  section  (labeled  MMl)  covers  students  entering  the  MM  school  before  the 
changes  of  January  7,  1991,  when  PE  and  MM  course  material  remained  distinct. 
This  section,  MM2,  addresses  students  entering  the  MM  school  after  the 
instructors  integrated  the  PE  and  the  MM  course  material  (as  of  January  7,  1991). 

Description  of  Variables.  The  school  determined  appropriate  item  weights 
fc  ■  converting  each  knowledge  test  grade  into  a  percentage  score.  See  Appendix 
G  for  a  list  of  these  weights.  The  school  requires  a  score  of  63%  or  higher  to 
pass  each  exam.  Students  who  fail  a  test  participate  in  instructor  oral 
remediation,  night  study,  and  retesting.  Examinees  cannot  retest  until  the  next 
academic  day.  All  students  who  fail  the  retest  appear  before  the  Academic  Review 
Board.  The  school  then  decides  upon  the  student’s  status.  The  instructor 
assigns  the  minimum  passing  score  to  those  students  who  pass  the  retest,  and  the 
school  uses  that  minimum  score  to  compute  the  final  school  grade  (FSG).  Although 
the  school  assigns  a  score  of  63  for  all  successful  retests,  R6I  entered  only  the 
initial  test  scores  for  each  examinee. 

The  MM  school  assesses  student  progress  using  knowledge  tests, 
comprehensive  tests,  and  quizzes.  Table  76  presents  these  variables  and  their 
corresponding  descriptions.  The  instructors  divide  the  curriculum  into  Phase  1 
and  Phase  2,  roughly  representing  a  focus  on  PE  and  MM  issues  (respectively). 
Phase  1  includes  tests  and  quizzes  up  to  and  including  the  first  comprehensive 
exam.  Phase  2  covers  the  remaining  half  of  the  course.  For  each  student,  the 
school  computes  a  Phase  1  and  a  Phase  2  grade  from  a  weighted  average  of  the 
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Tabic  7(3 


Descnriicins  of  MM2  Variables 


Test  No.  Description  (Name  -  Tv-pe  of  Test)^ 


204  Quiz  1:  Main  propulsion  plant/machincry  nomenclature,  shipboard  v^-aich  organization.  PQS  program,  shipboard  piping  systems. 

operating  principles  of  piping  sv-siem  components,  tracing  piping  systems,  heat  exchangers,  piping  system  components,  tag-out 
sv-siem  {'Piping  Sv-stem  -  O) 

305  Knowledge  Test  1:  Quiz  1  content  (Piping  S>'siem  -  K) 

206  Quiz  1  3-M  system  &  metal  fasteners  (3-M  System  -  Q) 

208  Quiz  3:  Common  hand  tools,  pipc/tubing  &  fitting,  lubricant,  packing/gaskets  &  insulation,  precision  measuring  (Tools  -O) 

309  Knc^dedge  Test  Z  Content  from  Quiz  2  and  Quiz  3  (3-M  &  Tools  -  K) 

212  Quiz  4;  Globc/ball/gate/necdlc/butterfly/plug*cock/chcck/rc!icf/seniincl  valves,  external  valve  inspection,  centrifugal  pumps, 
jet  pumps,  rotary'  pumps  (Valves  -  Q) 

313  Knowledge  Test  3:  Quiz  4  content  plus  cenirifugal/reciprocating  static  display  pumps  (Valves  -  K) 

214  Quiz  5:  Firemain  system,  auxiliary  machinery'  cooling  water  s\-stem.  waste  &  oily  water  sv-stems  (Water  S>-stcm  -Q) 

317  Knowledge  Test  4:  Quiz  5  content  plus  main  drain  ss-stem.  potable  water  system,  low  pressure  air  system,  communication 

devices,  single  agent  system,  haion  extinguishing  system,  bearings,  gears,  couplings.  &  plastics  control  (Water  System  -  K) 

319  Comprebcnsive  Knowledge  Test  1:  Propulsion  Engineering  Basics  issues  -  Content  from  tests  204  to  317  plus  introduction  to 
cold  iron  &  cold  iron  trainer  lab  (PE  Comp  -  C) 


223  Quiz  6:  Damage  control,  equipment  technical  manuals,  basic  steam  cycle,  boilers  (Steam  Cycle  -  Q) 

224  Quiz  7:  Fuel  oil  service  system  (Fuel  Oil  -  Q) 

325  Knowledge  Test  5:  Content  from  Quiz  6  and  Quiz  7  plus  combustion  air  (Steam  &  Fuel  -  K) 

227  Quiz  8;  Main  steam  system,  auxiliary  steam  system,  heat  stress  (Steam  System  -  Q) 

328  Knowledge  Test  6:  Quiz  8  content  plus  reduced  pressure  steam  &  superheater  protection  steam  (Steam  System  -  K) 

230  Quiz  9:  Main  Propulsion  engine  and  main  lube  oil  system  (Lube  System  -  Q) 

331  Knowledge  Test  7:  Quiz  9  content  plus  straincr/bearing  lab,  lube  oil  purification/iransfer  system  (Lube  System  -  K) 

334  Knowledge  Test  8;  Diaphragm  control  valve,  lube  oil  purifier,  gland  scal/gland  cxhaust/sea-waicr  circulating  system,  main 

condenser  (Gland  Seal  -  K) 

235  Quiz  10:  Main  condensate  system/main  air  removal  (Condensate  -  Q) 

237  Quiz  11:  Main  feed  system,  deaerating  feed  tank,  cuiaway/spray  nozzle  lab.  steam  drains  (Feed  System  -  Q) 

338  Knowledge  Test  9:  Quiz  10  &  11  content  plus  auxiliary  cxhaust/cscape  steam  piping  (Cond  &  Feed  -  K) 

239  Quiz  12:  Distilling  plant  (Distilling  -  Q) 

341  Knowledge  Test  ICk  Quiz  12  content  plus  make  up/cxccss/rcscivc  feed,  water  chemistry,  noise  awareness,  &  throttleboard 
stimulator  (Distilling  -  K) 

242  Quiz  13:  Basic  electricity  (Electricity  -  Q) 

344  KncMdedgc  Test  11:  Quiz  13  content  plus  ship  service,  turbo  generator,  electrical  distribution,  IC  alarm  (Electricity  -K) 

144  Peiformanoe  Test  (oral  board  comprebeosive  exam):  Basic  steam  cycle  (Steam  Cycle  -  P) 

349  Gomprebensive  Knowledge  Test  2:  Machinist's  Mate  issues  -  Content  from  tests  223  to  344  plus  Hot  plant  indoctrination,  main 

propulsion,  auxiliary  system  (MM  Comp  •  C) 


®  Q:  Quiz,  K: 


Knowledge, 


P:  Performance, 


C:  Comprehensive. 
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corresponding  test  scores  (see  Appendix  G  for  test  weights).  The  school  uses  the 
mean  of  the  two  phase  grades  as  the  FSG.  The  school  also  divides  the  curriculum 
into  eleven  units.  Each  knowledge  test  occurs  at  the  end  of  its  corresponding 
unit.  Although  quizzes  and  knowledge  tests  address  the  same  content  areas  within 
each  unit,  the  knowledge  tests  address  the  material  in  more  specific  terms.  All 
tests  and  quizzes  have  a  valid  range  of  0  to  100. 


Tabic  77 

Descnrnve  Statistics  for  MM2  \'anablcs 


Vanable 

T\ 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Piping  S\’Siem-0 

706 

80.167 

14.932 

-364 

-.315 

40.00 

Piping  S>’stcm-K 

706 

76.926 

09.290 

-.672 

1.110 

33.60 

098.20 

3-M  Svsicm-0 

705 

87.872 

11.445 

-1.504 

5.187 

10.00 

100.00 

Tools-O 

703 

84.669 

12.743 

.1.017 

2.122 

12.00 

100.00 

3-M  &  Tools-K 

703 

82,646 

09.212 

-.623 

.213 

52.90 

100.00 

VaKnes-Q 

700 

82.400 

14.401 

-.731 

.199 

20.00 

100.00 

Valvcs-K 

699 

79320 

12.837 

-307 

-.433 

32.70 

100.00 

Water  Svstem-O 

696 

83.250 

15,167 

-1.233 

2.621 

12.00 

100,00 

Water  S\’sicm-K 

694 

78.304 

12.189 

-356 

-.152 

32.70 

100.00 

PE  Comp-C 

692 

78.452 

10.447 

-313 

.284 

25.20 

100.00 

Steam  Cvclc-O 

690 

77.638 

17.302 

-.680 

.124 

20.00 

100.00 

Fuel  0i\0 

689 

81.480 

15.206 

-.682 

.016 

20.00 

100.00 

Steam  6l  FucI-K 

688 

74.628 

13.848 

-.420 

-.285 

25.20 

100.00 

Steam  Sv'siem-O 

685 

92.321 

09.942 

-1.463 

2.148 

50.00 

100.00 

Steam  System-K 

685 

74386 

13.429 

-.252 

-.404 

28.60 

100.00 

Lube  System-Q 

683 

77.088 

17.897 

-.683 

.129 

10.00 

100.00 

Lube  Sysiem-K 

682 

75.835 

11.874 

-310 

.812 

1430 

100.00 

Gland  ^a!-K 

680 

75.991 

13391 

-.463 

-.146 

28.40 

100.00 

Condensate-Q 

680 

80.174 

16386 

-.751 

.182 

12.00 

100.00 

Feed  System-Q 

676 

77.707 

17352 

-.778 

.292 

20.00 

.  100.00 

Cond  &  Feed-K 

678 

7U15 

13.185 

-.292 

-.108 

1930 

100.00 

Distilling-Q 

673 

81.070 

16.194 

-.962 

.657 

20.00 

100.00 

Distilling'K 

669 

75.698 

12.463 

-.379 

-.112 

25.80 

100.00 

Elect  ricity-Q 

667 

87370 

12.761 

-1.157 

1,413 

30.00 

100.00 

Elcctricity-K 

665 

79.193 

11.911 

-.473 

-.353 

37.20 

100.00 

Steam  Cycle-P 

604 

100.000 

100.000 

. 

. 

100.00 

100.00 

MM  Comp-C 

662 

74358 

09,749 

-.116 

-.365 

4730 

100.00 
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Sample.  The  initial  sample  consisted  of  710  students  who  began  training 
between  January  and  October  1991.  We  excluded  students  who  did  not  take  every 
test  from  some  of  the  analyses.  Consequently,  our  final  sample  contained  659 
cases. 

Results  and  Discussion.  RGI  computed  descriptive  statistics  for  each  of 
the  test  variables.  Table  77  presents  these  statistics  for  all  variables.  The 
table  lists  an  n  of  604  cases  for  the  performance  test  at  the  end  of  Phase  2 
(Steam  Cycle-P).  Because  106  students  were  missing  data  for  this  test,  it  was 
not  included  in  some  of  the  analyses.  However,  the  MM  school  includes  this  test 
when  calculating  the  Phase  2  grade  and  the  FSG.  Consequently,  we  maximized  our 
n... Tiber  of  cases  when  computing  Phase  2  and  FSG  scores  by  prorating  scores  on  the 
performance  test  for  the  106  cases.  This  involved  taking  the  average  of  their 
available  scores  and  weighting  the  remaining  tests  appropriately. 

Next,  we  conducted  a  factor  analysis  on  the  set  of  all  test  variables.  The 
scree  test  and  the  factor  loadings  indicated  one  factor.  Table  78  shows  the 
communal i ties,  factor  loadings,  and  factor  score  coefficients  for  each  of  the 
variables.  Factor  loadings  ranged  from  .256  to  .673. 

We  sought  to  develop  composite  variables  which  would  capture  essential 
elements  of  the  criterion  variance  and  also  represent  meaningful  content 
dimensions  useful  to  the  MM  school.  Since  the  school  divides  the  curriculum  into 
phases  and  computes  grades  for  each  phase,  we  used  the  school’s  weighting  system 
to  compute  Phase  1  and  Phase  2  grades.  We  also  created  composites  based  on  test 
type  by  averaging  all  quizzes  for  one  composite  and  all  knowledge  tests  for 
another.  Since  the  MM  school  computes  the  FSG  using  63  as  the  minimum  score,  RGI 
calculated  its  own  final  grade  (FSG2)  using  initial  test  scores  and  employed  both 
as  composite  variables.  We  used  the  factor  score  (Factor  1)  as  an  additional 
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composite.  Table  79  presents  descriptive  statistics  for  each  of  the  composite 
variables  mentioned  above.  We  placed  all  of  the  composites  on  a  0-100  scale  in 
order  to  facilitate  comparing  them.  The  sample  size  varied  for  the  composite 
variables  depending  on  the  number  of  tests  missing  for  each  case. 


';ible  7S 


Communaiities.  Factor  Loadings  and  Factor  Score  Coefficients  for  MM2  Variables 


Variable 

Communaiity 

Factor  Loadings 

Faaor  Score  Coefficients 

Piping  S>'siem-0 

.195 

.442 

.046 

Piping  S>-sicm-K 

.397 

,630 

.093 

3-M  Sv’sicm-O 

.066 

.256 

.026 

Tools^ 

.120 

.346 

.036 

3-M  &  Tcx)ls-K 

.319 

J65 

.076 

Valves-O 

.152 

.390 

.042 

V’alvw-K 

.343 

.586 

,073 

Water  Sv’siem-O 

.103 

.321 

.033 

Water  S\-stem-K 

.389 

.624 

.091 

PE  Comp-C 

.344 

SSI 

.083 

Steam  Cycle-Q 

.211 

.460 

.050 

Fuel  0\\0 

.221 

.470 

.055 

Steam  di  Fuel-FC 

.453 

.673 

.109 

Steam  Sv-siem-Q 

.107 

.327 

.034 

Steam  Si.’stcm-K 

.387 

.622 

.085 

Lube  S>-siem-0 

.159 

.399 

.051 

Lube  Sv'stcm-K 

.397 

.630 

.092 

Gland  Scal-K 

.330 

.574 

.074 

Condensate-Q 

.225 

.474 

.058 

Feed  System-O 

.105 

.324 

.034 

Cond  &  Fccd-K 

.428 

.654 

.098 

DistillingO 

.244 

.494 

.060 

Distilling-K 

.253 

S03 

.058 

Electricity-Q 

.180 

.425 

.046 

Electricity- K 

.357 

S91 

.081 

MM  Comp-C 

.409 

.639 

.092 

Note.  N  »  659. 

Table  79 

DescnDtivc  Statistics  for  MM2  Comnosiie  Variables 

Composite  n  Rcl^  Mean 

Std  Dev 

Skew  Kurtosis  Min 

Max 

FSG 

707 

.876 

79i31 

6.583 

o 

t 

1.730 

40.30 

96.80 

FSG2*’ 

659 

N/A 

78.415 

7.047 

.105 

-.375 

57.16 

.  96.77 

KTcst= 

661 

.885 

77.010 

7J85 

.130 

-.366 

55.94 

96.95 

Quiz*^ 

664 

.739 

82.815 

7.171 

-360 

-.207 

60.77 

9933 

Phasel* 

690 

.742 

79.600 

7.572 

-.137 

.467 

57.10 

96.25 

Phasc2^ 

660 

.837 

76.933 

7.822 

.105 

.435 

54.93 

9738 

a  These  are  composites  based  on  conxnunaiity  estimates  and  the  procedures  described  in  the  text, 
b  FSG2:  Final  School  Grade  computed  using  initial  scores, 
c  Ktests:  Average  of  all  knowledge  test  scores, 
d  Quiz:  Average  of  all  quiz  scores, 
e  Phasel:  See  Appendix  _  for  computation, 
f  Fhase2;  See  Appendix  _  for  computation. 
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Finally,  we  computed  Pearson  correlation  coefficients  for  the  composites 
Results  revealed  high  zero-order  correlations  between  all  composite  variables 
Table  80  presents  these  correlations. 

Recommendations.  RGI  recommends  using  the  school -calculated  FSG. 


Table  80 

Corretations  Among  .MM2  Composites  and  Factors 


Comp  Fad  FSG  FSG2® 


FSG 

.97 

fsg: 

.98 

.99 

Kiest 

.98 

.97 

.99 

Quiz 

.86 

.77 

.76 

Phasel 

.88 

.93 

.92 

Phase’® 

.94 

.91 

.93 

Ktcst^ 


.74 

.87  .70 

.96  .72 


Phase 


.72 


^FSG2;  Final  Schcx)l  Grade  computed  using  initial  test  scores. 

Ktests;  Average  of  all  knowledge  test  scores. 

^Quiz:  Average  of  all  quiz  scores. 

Phasel:  See  Appendix  for  computation, 

Phasc2:  See  Appendix  for  computation. 
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Operations  Specialist  (OS) 

Description  of  Variables.  The  OS  school  evaluates  students  using  written, 
performance,  and  blitz  tests.  The  school  modified  the  test  forms  in  August  1990. 
Test  Form  A  refers  to  the  previous  test  form,  and  Test  Form  B  refers  to  the 
current  test  form  in  use.  Although  changes  occurred  in  the  sequence  and  the 
number  of  tests,  the  test  content  did  not  change.  In  some  cases,  the  school 
consolidated  the  performance  and  written  tests  into  one  test.  Notes  throughout 
the  variable  descriptions  indicate  the  occurrence  of  a  sequence  change  or  a 
consolidation  of  test  material.  Where  sequence  changes  occurred  between  test 
forms,  we  placed  the  form  associated  with  each  test  name  in  parentheses  (e.g.. 
Form  B  only).  We  designated  written  test  scores  as  (W),  performance  test  scores 
as  (P),  and  tests  consisting  of  a  performance  test  and  a  written  test  as  (P/W). 
Table  81  provides  the  description  and  sequence  of  the  Form  A  and  Form  B  OS  tests. 

The  school  sets  the  minimum  passing  score  for  each  test  at  70.  For 
students  passing  the  retest,  the  OS  school  assigns  a  score  of  70  to  be  included 
in  the  student’s  final  school  grade  (FSG).  We  entered  only  the  students’  initial 
scores  into  the  database. 

The  FSG  currently  consists  of  a  simple  average  of  all  performance,  written, 
and  performance/written  exams.  The  school  did  include  blitzes  (quizzes)  in  the 
final  average  until  May  1991.  Instructors  now  use  blitzes  only  as  a  tool  to 
evaluate  the  weekly  progress  of  the  students.  The  school  decided  that  including 
the  blitzes  in  the  FSG  conflicted  with  the  purpose  of  the  blitzes,  identifying 
problem  areas  where  students  need  additional  training.  We  decided  not  to  include 
the  blitz  scores  in  the  analyses  for  three  reasons.  First,  including  both  tests 
and  blitzes  in  the  analyses  causes  multi  col  linearity  problems  because  they  cover 
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TjMc  M 


ng^cnrtions  of  OS  Variables 


\anablcs 


Secunty/Bcanngs 


Basic  Mancuvcnng 
Board 

Adv:anccd  Mancuvcnng 
Board  I 


Dcscnpijon  (Abbreviated  Name  -  Tvpc  of  Test  ) 

Tests  knowledge  on  procedures  for  handling  classified  maienal  and  procedures  for  convening  True,  Relative, 
and  Reciprocal  Bearings  and  Target  Angles.  (Security  -  W) 

Tests  knowledge  and  ability  to  solve  relative  motion  problems  on  a  Mancuvcnng  Board  for  cither 
ownship  or  contact  course,  speed,  and  closest  point  of  approach.  (Basic  Man  Bd.  -  P/W) 

Tests  knowledge  and  abiliw  to  determine  a  revised  closest  point  of  approach  and  to  compute  ownship 
course  change  to  avoid  a  contact  by  a  specified  distance.  (Adv  Man  Bdl  -  P) 


(The  school  flip-flopped  the  sequence  of  the  following  two  tests  on  the  two  forms.) 


Internal 

Communications 

Maneuvenng 
Board  2 


.Adv-anced  Manucuvering 
Board  2 


Internal 

Communication/ 

3  Minute  Rule 

Publications/Logs 


Tests  knowledge  and  ability  to  communicate  with  another  station  on  sound  powered  phone  circuits.  (Test 
Form  A  only)  (Int.  Comm.  -  P/W) 

Tests  knowledge  and  ability  to  compute  on  ownship’s  maneuver  to  change  its  assigned  station.  Tests 
knowledge  and  ability  to  use  the  maneuvering  board  to  determine  the  apparent,  relative,  and  true  direction 
and  speed  of  the  wind.  (Test  Form  A  only)  (Man.  Board2  -  P) 

Tests  knowledge  and  ability  to  compute  on  ownship’s  maneuver  to  change  its  assigned  station.Tests 
knowledge  and  ability  to  use  the  mancMvenng  board  to  determine  the  apparent,  relative,  and  true  direction 
and  speed  of  the  wind.  (Test  Form  B  only)  (Adv  Man  Bd2  -  P) 

Tests  knowledge  and  ability  to  communicate  with  another  station  on  sound  powered  phone  circuits  and  use 
the  3  minute  rule  to  solve  for  speed,  (Test  Form  B  only)  (Int.  Comm/3  Min  -  P/^^0 

Tests  knowledge  and  ability  to  maintain  a  Combat  Information  Center  (CIC)  Log  using  proper 
abbreviations.  Tests  knowledge  and  ability  to  use  proper  CIC  publications.  (Pubs/Logs  -  P/W) 


(The  school  consolidated  the  content  of  the  following  two  Form  B  tests  into  other  version  A  tests.  The  sch<x)l  included  the  content  of 
Mancuvcnng  Board  3  into  the  Adv-anced  Maneuvenng  Board  2  test,  and  the  content  of  the  3  minute  rule  with  the  Internal  Communication 
test.) 


Mancuvcnng  Board  3 


3  Minute  Rulc/S>mbols 


Signal  Book  Formation 


Cartesian  Grid 

External 

Communication 


Tests  knowledge  and  ability  to  compute  on  ownship’s  maneuver  to  change  its  assigned  station  and  use  the 
maneuvenng  board  to  determine  the  apparent,  relative,  and  true  direction /speed  of  wind.  (Test  Form  A 
only)  (Manuv.  Board3  -  P) 

Tests  knowledge  and  ability  to  use  3  minute  rule  to  solve  for  speed.  (Test  Form  A  only)  (3Min/Symbols  - 
W) 

Tests  knowledge  and  ability  to  encode  and  decode  given  signals,  and  use  Signal  Book  to  put  ships  in 
formation.  (Signal  Bk  -  P) 

Tests  knowledge  and  ability  to  plot  contacts  on  a  Cartesian  Grid.  (Cart  Grid  -  P) 

Tests  knowledge  and  ability  to  communicate  with  another  ship  on  a  radiotelephone  circuit.  (Ext.  Comm. 
P/W) 


(The  school  flip-flopped  the  sequence  of  the  following  two  tests  on  the  two  forms.) 


Air  Plotter 


Tests  knowledge  and  ability  to  plot  contacts  with  amplifying  information  on  an  Air  Summary  Board. 
(Test  Form  A)  (Air  Plot  -  P/W) 


®Key  to  type  of  test:  W-Written;  P-Performance;  P/W-Pcrformancc/Wntten 
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Table  hi  (Coniinucd) 


\’anablc  Dcscnpiion  t  Abbreviated  Name  -  Tvpc  oi  Tesi*^) 


Surface  Summarv* 

Surface  Summary 

.Air  Plotter 

Dead  Reckoning 
Tracer 

Chans  Sea  &  Anchor 

Coastal  Navigation 


67/25/250/ 

Rules  of  the  Road 
(ROR^ 


SPA.25F  PT 


Tests  knowledge  and  ability  to  plot  contacts  with  amplifying  information  on  a  Surface  Summary  Board. 
(Test  Form  A  only)  (Surf.  Summ,  -  P) 

Tests  knowledge  and  ability  to  plot  contacts  with  amplifying  information  on  a  Surface  Summary  Board. 
(Test  Form  B  only)  (Surf.  Summ.  -  P) 

Tests  knowledge  and  ability  to  plot  contacts  with  amplifying  information  on  an  Air  Summary  Board. 
(Test  Form  B  only)  (Air  Plot  -  P/W) 

Tests  knowledge  and  ability  to  perform  duties  of  a  Dead  Reckoning  Tracer  operator  using  DRT 
plotting  procedures,  symbology  and  abbreviations.  Tests  knowledge  and  ability  to  compute  course  and 
speed  on  two  surface  contacts.  (Dead  Rec.  -  P) 

Tests  knowledge  of  Chart  Correction  Card  System  and  purpose  of  the  Notice  to  Mariners.  Tests  the 
ability'  to  locate  chans  by  name  and  number  for  transit  situations.  (Charts  -  W) 

Tests  knowledge  and  ability  to  perform  duties  of  the  Navigation  Plotter,  to  plot  coastal  navigation  fixes, 
to  plot  dead  reckoning  positions  along  intended  track,  to  compute  course  and  speed  made  good 
between  two  navigation  fixes,  to  compute  set  and  drift  between  a  dead  reckoning  position  and  a 
navigation  fix.  to  compute  and  plot  an  estimated  position,  and  to  plot  an  operating  area  and  to  record 
navigational  fix  data.  (Coast  Nav.  -  p) 

Tests  knowledge  and  ability  to  perform  duties  of  Radar  Plan  Position  Indicator  Operator,  detect,  plot, 
and  repon  all  surface  or  air  contacts,  distinguishing  them  from  weather,  land,  and  sea  return.  Tests 
ability  to  obtain  CP.A  (closest  point  of  approach)  bearing  range  and  time,  and  course  and  speed  on 
three  surface  contacts.  Tests  knowledge  of  the  SPS/67  Radar  Set  Control.  Tesu  knowledge  of  an 
overtaking,  crossing,  and  meeting  situation,  and  the  purpose  and  types  of  navigation  lights.  (Rules  Rd.- 
W) 

Tests  ability  to  compute  revised  CPA  (bearing,  range,  and  time),  to  solve  for  CPA  for  each  contact,  and 
to  perform  duties  of  Radar  Scope  Operator,  detect,  plot  and  report  all  surface  and  air  contacts.  (Radar 
Scope  Op  -  P) 


(The  school  consolidated  the  following  Air  and  Surface  NTDS  WTN  tests  and  the  Air  and  Surface  NTDS  PT  tests  from  Form  B  into  a 
single  written  test  and  a  single  performance  test,  respectively,  on  Form  A.) 


Air  NTDS 

Air  NTDS 


Surface  NTDS 


Tests  knowledge  of  the  duties  of  Naval  Tactical  Data  System  Input  Console  (NTDS)  Operator  in  the 
Air  Tracker  mode  of  operation.  (Test  Form  A)  (Air  Tracker  •  ^ 

Tests  ability  to  perform  duties  of  NTDS  Input  Console  Operator  in  the  Air  Tracker  mode  of  operation. 
(Test  Form  A  only)  (Air  Tracker  -  P) 

Tests  ability  to  perform  duties  of  NTDS  Input  Console  Operator  in  the  Surface  Tracker  mode  of 
operation.  (Test  Form  A  only)  (Surface  Tracker  -  P) 


Surface  NTDS 


NTDS 

NTDSPT 


Tests  knowledge  of  the  duties  of  NTDS  Input  Console  Operator  in  the  Surface  Tracker  mode  of 
operation.  (Test  Form  A  only)  (Surface  Tracker  -  W) 

Tests  knowledge  of  the  duties  of  NTDS  Input  Console  Operator  in  the  Surface  Tracker  and  Air 
Tracker  mode  of  operation.  (Test  Form  B  only)  (Air/Suif  Tracker  -  W) 

Tests  ability  to  perform  duties  of  NTDS  Input  Console  Operator  in  the  Surface  Tracker  and  Air 
Tracker  mode  of  operation.  (Test  Form  B  only)  (Air/Surf  Tracker  -  P) 


*Key  to  type  of  test:  W-Written:  P-Performance:  P/W-Performancc/Writtcn 
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the  same  material  as  the  tests.  Second,  the  motivational  impact  of  a  blitz 
differs  from  a  test  because  students  realize  their  blitz  performance  will  not 
affect  their  FSG.  Finally,  the  school  ultimately  excluded  them  from 
performance  assessment  and  used  them  more  as  a  training  tool . 

Sample.  The  total  sample  consisted  of  802  students.  The  school  used 
the  older  Form  A  for  153  of  these  students.  Of  these  153  cases,  109  had 
completed  all  Form  A  tests.  The  remaining  649  students  completed  Form  B.  We 
received  complete  data  for  530  of  these  cases.  We  excluded  cases  with  missing 
data  from  some  of  the  analyses. 

Results  and  Discussion.  We  began  by  testing  the  two  groups  of  students 
for  differences  due  to  the  change  in  curriculum.  We  conducted  a  multivariate 
analysis  of  variance  (MANOVA)  to  test  the  correlation  matrices  of  the  test 
scores.  A  test  for  homogeneity  of  covariance  matrices  revealed  that  the  two 
forms  had  significantly  different  covariance  matrices  (F=2.152,  fi<.001).  The 
test  means  also  differed  significantly  across  the  two  forms  (F=11.553  fi<.001). 

To  test  whether  ability  accounted  for  these  differences  between  the  two 
groups  of  students,  RGI  conducted  a  multivariate  analysis  of  covariance 
(MANCOVA)  using  the  student’s  pre-enlistment  ASVAB  scores  as  covariates. 

After  parti  ailing  out  the  variance  due  to  ASVAB  scores,  test  for  homogeneity 
of  covariance  matrices  revealed  different  matrices  for  the  two  forms  (F=1.53, 
£<.001).  Test  means  also  differed  across  the  two  forms  after  controlling  for 
ASVAB  scores  (F=11.985,  fi<.001).  Univariate  F-tests  indicated  that  8  of  the 
18  variables  differed  between  forms  at  the  e<.05  level.  These  results  suggest 
that  the  noted  difference  between  the  two  groups  is  due  to  something  other 
than  differences  in  ability.  Based  on  these  findings,  we  analyzed  each  group 
separately. 
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To  further  evaluate  the  conclusion  of  differences  between  forms,  we 
computed  effect  sizes  as  measures  of  the  practical  significance  of  the 
difference  between  the  unadjusted  means  across  forms.  Cohen  and  Cohen  (1977) 
say  that  .2  represents  a  small  effect  size,  .5  is  medium,  and  .8  is  large. 

For  the  total  of  18  comparisons  among  the  OS  forms,  we  found  10  within  the 
range  of  .00  -  .19,  6  within  the  range  of  .20  -  .49,  1  within  the  range  of  .5 
-  .79,  and  1  greater  than  .80.  These  results  show  that  89%  of  the  effect 
sizes  fall  below  a  value  of  .50.  Thus,  while  the  MANOVA  tests  indicate  that 
the  forms  differ  from  one  another  in  terms  of  statistical  significance,  the 
effect  sizes  suggest  small  magnitudes  of  the  differences,  in  terms  of 
practical  significance. 

We  computed  descriptive  statistics  for  each  of  the  test  forms 
separately.  Table  82  presents  these  statistics  for  Form  A  and  Form  B. 

After  viewing  score  distributions,  we  conducted  a  factor  analysis  on 
each  form  separately.  In  Form  A  the  scree  test  and  factor  loadings  indicated 
a  one  factor  solution.  The  Form  B  scree  test  and  the  factor  loadings 
indicated  two  factors.  Based  on  the  way  variables  loaded  on  the  two  Form  B 
factors,  we  labeled  the  first  factor  Communication  and  the  second  factor 
Navigation/Plotting  (Nav/Plot).  Table  83  lists  the  commonalities,  factor 
loadings,  and  factor  score  coefficients  for  each  of  the  Form  A  and  the  Form  B 
variables. 

Next,  we  created  various  composite  variables  for  Form  A  and  Form  B.  In 
order  to  examine  whether  a  communication  versus  navigation/plotting  test 
content  distinction  could  explain  aspects  of  school  performance,  we  created 
the  composite  variables  of  "Communication"  and  "Navigation/Plotting"  for  Form 
B.  We  computed  these  by  taking  an  average  of  those  tests  with  a  description 
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Table 


Ocscrinive  Statistics  for  OS  \'anables 


Variable 

n 

Mean 

Std  Dev  Skew 

Kurtosis  Min 

Max 

Form  A 


Sccunrv  -  W 

151 

89.457 

9.792 

-1.613 

3.157 

48.00 

100.00 

Basic  Man  Bd.  -  P/'W' 

153 

89.216 

8.299 

-.975 

320 

6230 

100.00 

Adv  Man  Bdl  -  P 

153 

83.327 

12.248 

-1.203 

1.671 

3730 

100.00 

Man  Board2  -  P 

153 

84.072 

12.690 

-  .996 

1.141 

40.00 

100.00 

Man  Boards  -  P 

153 

88.556 

11.822 

-1.011 

.381 

50.00 

100.00 

Int.  Comm  -  PfV/ 

153 

91.405 

6.974 

-1.253 

2.105 

63.00 

100.00 

3Min/S\*mbols  -  P/V*' 

153 

93.575 

7.849 

-1.976 

5.732 

52.00 

100.00 

Pubs/Logs  -  P/V>' 

153 

89.627 

6.766 

-  .809 

.325 

70.00 

100.00 

Signal  Bk  -  P 

153 

86.993 

10.344 

-1.048 

1.233 

48.00 

100.00 

Can  Gnd  -  P 

153 

93i36 

7.289 

-2.367 

7.804 

48.00 

100.00 

Ext  Comm.  •  ?P^’ 

153 

88,784 

6.849 

-1,017 

1.784 

59.00 

100.00 

Surf  Summ.  A  -  P 

153 

92.984 

7.911 

-3.350 

15.428 

44.00 

100.00 

.Air  Plot  A  -  P/W 

153 

90.748 

6.965 

-1.367 

1.625 

70.00 

100.00 

Dead  Rec.  -  P 

153 

88.016 

8335 

-1.085 

1.392 

54.00 

100.00 

Ch  ns  - W 

153 

85.978 

8323 

-.425 

-  366 

64.00 

100.00 

Coast  Nav.  .  P 

153 

86.418 

10.170 

-  .829 

.512 

52.00 

100.00 

Rules  Rd.  -  W 

153 

84342 

8.333 

-  .526 

.193 

55.00 

100.00 

Radar  Scope  Op  -  P 

153 

98.804 

1.950 

-2.050 

4.959 

90.00 

100.00 

Surface  Tracker  -  W 

150 

82.899 

9342 

-.287 

.288 

55.00 

100.00 

Surface  Tracker  -  P 

150 

92.968 

6.841 

-1.299 

1.737 

70.00 

100.00 

Air  Tracker  -  W 

115 

81.967 

8.717 

-  .045 

-  .700 

58.00 

100.00 

Air  Tracker  -  P 

114 

94313 

11.114 

-3.053 

12.090 

2730 

100.00 

Foim  B 


Security  •  W 

649 

85.637 

9366 

-  .872 

1.349 

3730 

100.00 

Basic  Man  Bd.  -  P/W 

649 

85305 

11.852 

-  .934 

.925 

36.00 

100.00 

Adv  Man  Bdl  -  P 

649 

81.715 

13.811 

-1.154 

1.854 

12.00 

100,00 

Adv  Man  Bd2  -  P 

649 

89.445 

10.865 

-1.461 

2.675 

36.00 

100.00 

Int.  Comm/3  Min  -  P/W 

649 

92,242 

5,618 

-.813 

.456 

70.00 

100.00 

Pubs/Logs  •  P/W 

649 

89.675 

7.199 

-.933 

1.003 

57.00 

100.00 

Signal  Bk  -  P 

649 

83.324 

11.034 

-.835 

.875 

34.00 

100.00 

Can  Grid  -  P 

649 

94.018 

7.013 

-2.237 

7,807 

4130 

100.00 

Ext  Comm.  -  P/W 

649 

93.076 

5.884 

-1.214 

1,930 

62.00 

100.00 

Surf  Summ  B  -  P 

649 

91.444 

7.766 

-1.140 

.845 

6030 

100.00 

Air  Plot  B  -  P 

646 

90.457 

7.007 

-1368 

2.255 

58.00 

100.00 

Dead  Rec.  -  P 

648 

89.746 

7.746 

-1.099 

.988 

5930 

100.00 

Charts- W 

647 

85.146 

9.477 

-.886 

1.037 

45.00 

100.00 

Coast  Nav.  -  P 

643 

87.409 

11.177 

-1.440 

2.682 

38.00 

100.00 

Rules  Rd.  -  W 

643 

83.063 

8.986 

-.219 

-.446 

55.00 

100.00 

Radar  Scope  Op  -  P 

643 

98390 

3.472 

-4.9V 

36.627 

60.00 

100.00 

Air/Surf  Tracker  -  W 

596 

85.702 

7320 

-.465 

.471 

48.20 

100.00 

Air/Surf  Tracker  -  P 

548 

87.847 

10.212 

-1.284 

2367 

36.80 

100.00 
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Tabic  b5 


Communatities.  Factor  Loadings  and  Factor  Score  Cocfficienis  for  OS  Variables 


Form  A  \‘ariablc 

Communaiit\' 

Factor  Loadings 

Factor  Score  Coefficients 

Sccuniy  -  W 

.232 

.482 

.082 

Basic  Manuv.  Bd  -  P/W 

.473 

.687 

.181 

Adv.  Man  Bdl  -P 

.220 

.469 

.072 

Manuv.  Bd2  -  P 

.304 

SSI 

.100 

Manuv.  Bd3  -  P 

.128 

.358 

.050 

Int  Comm  -  PA^^ 

.358 

-599 

.101 

3Min  Rulc/S™  -  P/'V^' 

.181 

,425 

.069 

Pubs /Logs  -  P/W 

.381 

.617 

.144 

Signal  Bk  -  P 

.198 

.445 

.060 

Cart  Grid  -  P 

.253 

-503 

.086 

Ext.  Comm  -  PfW 

.256 

S06 

.079 

Surf  Summ  -  P 

.212 

.461 

.072 

Air  Plot  -  P  'W 

.317 

i64 

,130 

Dead  Rec.  -  P 

.342 

i84 

.106 

Chans  -  W 

,179 

.424 

.072 

Coast  Nav.  -  P 

.203 

.450 

.069 

Rules  Rd.  -  W 

.178 

.422 

.064 

Radar  Scope  Op  -  P 

.017 

.132 

.019 

Surf  Tracker  -  W 

.085 

.292 

.038 

Surf  Tracker  -  P 

.040 

.199 

.021 

Air  Tracker  -  W 

.246 

.495 

.075 

Air  Tracker  -  P 

.005 

.071 

.002 

Factor  Loadings  Factor  Score  Coefficients 


Form  B  Variable 

Communality' 

Factor  1: 
Communication 

Factor  2: 
Nav/Plot 

Factor  1: 
Communication 

Factor  2: 
Nav/Plot 

Security’  -  W 

.365 

543 

.264 

.147 

.006 

Basic  Man  Bd.  -  P/W 

.201 

.289 

.343 

.032 

.079 

Adv.  Man  Bdl  -P 

.248 

.310 

.390 

.030 

.095 

Adv.  Man  BD2  -  P 

.130 

.263 

.248 

.034 

.044 

Int  Comm/3  Min  -  P/W 

.437 

595 

.287 

.188 

.004 

Pubs/Logs  .  P/W 

.309 

508 

.226 

.139 

-.004 

Signal  Bk  -  P 

.326 

.452 

.350 

.096 

.063 

Cart  Grid  -  P 

.325 

.224 

524 

-.024 

.183 

Ext.  Comm  -  P/W 

.229 

.433 

.201 

.103 

-.002 

Surf  Summ  -  P 

.390 

.076 

.620 

-.110 

.284 

Air  Plot  -  P/W 

.439 

.070 

.659 

-.133 

.336 

Dead  Rec.  -  P 

.280 

.240 

.471 

-.006 

.154 

Charts  -  W 

.468 

.662 

.171 

.263 

-.077  , 

Coast  Nav.  -  P 

.259 

.369 

.351 

.055 

.084 

Rules  Rd.  •  W 

.308 

552 

.055 

.185 

-.085 

Radar  Scope  Op  -  P 

.061 

.103 

.224 

-.009 

.057 

Air/Surf  Tracker  -  W 

.330 

557 

.141 

.181 

-.053 

Air/Surf  Tracker  -  P 

.105 

.167 

.277 

.007 

.062 

Note.  Form  A;ji=109,  Form  B;  ji  -  471 
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related  to  the  content  of  the  variable.  Additional  Form  B  composites  included 
the  Suml  and  Sum2  variables  which  took  a  simple  average  of  all  the  tests  that 
loaded  .30  or  above  on  either  Factor  1  or  Factor  2,  respectively.  We  also 
used  the  Form  B  factor  scores,  Factorl  and  Factor2,  as  composite  variables. 

For  Form  A,  we  created  a  Suml  composite  variable  which  took  a  simple  average 
of  all  tests  that  loaded  .30  or  above  on  Factor  1  and  also  used  the  Form  A 
factor  score,  Factorl,  as  a  composite  variable.  For  both  Form  A  and  Form  B, 
we  combined  all  written  tests,  all  performance  tests,  and  all 
performance/written  tests  separately  to  determine  whether  or  not  the  type  of 
test  administered  could  explain  school  performance.  Because  the  OS  school 
splits  the  training  into  two  parts,  we  created  two  phase  variables  for  both 
forms  based  on  this  separation.  The  first  phase  includes  all  tests  up  to  and 
including  the  Air  Plot  test.  The  second  phase  consists  of  the  remaining 
tests.  Because  the  OS  school  computes  the  FSG  using  a  70  as  the  minimum 
score,  we  calculated  our  own  Final  School  Grade  (FSG2)  using  initial  test 
scores,  and  listed  both  as  composite  variables  for  both  forms.  Table  84  lists 
the  descriptive  statistics  for  each  of  the  composite  variables  listed  above. 
The  n  varied  for  the  composite  variables  depending  on  the  number  of  tests 
missing  for  each  case. 

Finally  we  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Tables  85  and  86  present  the  intercorrelations  between  all 
composite  variables  for  Forms  A  and  B,  respectively.  Notice  that  FSG  and  FSG2 
correlated  .99  showing  no  support  for  using  initial  test  scores  to  compute 
FSG.  For  Form  B,  these  results  also  show  that  FSG  correlates  highly  with 
Communication  (.90)  and  Navigation/Plotting  (.91),  Performance  (.91)  and 
Phasel  (.96). 
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Descnrtive  Statistics  for  OS  ComTX)sue  Variables 


Composiic^  n  Rel* 


Mean  Std  Dev  Skew  Kurtosis  Min  Max 


Form  A 


Suml^ 

113 

.S6 

FSG 

150 

N7A 

FSG2‘= 

109 

.85 

Writ'* 

no 

.54 

Perf* 

114 

.68 

153 

.69 

Phase is 

151 

.S3 

Phase2^ 

111 

87.961 

4.912 

-.730 

88.043 

4.296 

-300 

88.761 

4.313 

-.605 

84.891 

5.444 

-.307 

89.850 

4.844 

a 

do 

90-559 

4.860 

-.888 

89.387 

5.188 

-.883 

88380 

4.064 

-.218 

326 

7330 

98.36 

.074 

75.67 

97.31 

.112 

77.02 

97.48 

-.356 

71.00 

97.20 

.486 

76.41 

97.73 

377 

74.67 

99.42 

.761 

73.73 

99.42 

-.676 

7830 

96.39 

Form  B 
Suml* 

Sum2'' 

FSG 

i  mm  * 

Plot^ 

Writ” 

Pcrf^ 

,  Perf/WritP 
Phascl^ 

Phasc2^ 

^These  are  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 

Suml:  Average  of  Security,  Basic  Man  Bd,  adv  Man  Bdl.  Man  Board2,  Man  Board  3e,  Int  Comm.  3Min/Symbols.  Pubs/Logs,  Signal 
Bk.  Cart  Grid.  Ext  Comm,  Surf  Summ.  Air  Plot.  Dead  Rec.  Charts.  Coas.  Rules  Rd.  Air  Tracker  -  W. 

^FSG2;  Final  School  Grade  using  initial  test  scores. 

^Writ:  Average  of  all  written  tests. 

®Perf:  Average  of  all  performance  tests. 

^Perf/Writ:  Average  of  all  performance /wntten  tests. 

*Phascl:  Average  of  tests  Security  to  Air  Plot. 

^Phase2:  Average  of  tests  Dead  Rec  to  Air  Tracker  -  P. 

^Suml:  Average  of  tests  Security,  Adv  Man  Bdl.  Int  Comni/3Min.  Pubs/Logs,  Signal  Bk.  Ext  Comm,  Charts.  Coast  Nav,  Rules  Rd 
Air/Surf  Tracker. 

JSum2:  Average  of  tests  Basic  Man.  Bd,  Adv.  Man  Bdl,  Signal  Bk,  Can  Grid,  Surf  Summ,  Air  Plot,  Dear  Rec,  Coast  Nav. 

*FSG2:  Final  School  Grade  using  initial  test  scores. 

^mm:  Average  of  all  tests  associated  with  Communication  (Security,  Int.  Comm/3M.  Pubs/Logs,  Signal  Bk.,  Ext.  Comm.,  Charts 
“Tlot:  Average  of  all  tests  associated  with  Plotting  (Cart.  Grid,  Surf  Summ.  Air  Plot,  Dead  Rec,  Coast  Nav,  Rules  Rd,  Radar  Scope 
Op,  Basic  Man  Bd,  Adv.  Man  Bdl.  Adv.  Man  Bd2.  ^ 

”Writ:  Average  of  all  written  tests. 

®Perf:  Average  of  all  performance  tests. 

PPerf/Writ:  Average  of  performance /written  tests. 

*^Phasel:  Average  of  tests  Security  -  Air  Plot. 

^Phase2:  Average  of  tests  Dead  Rec  -  Air/Surf  Tracker-P. 


5S7 

.82 

86.647 

639 

"7 

87.878 

711 

.\/A 

88.428 

530 

.85 

88313 

639 

.77 

87.422 

633 

.74 

89.092 

589 

.79 

84.802 

540 

.71 

89.226 

646 

.68 

90.165 

646 

.81 

88.745 

533 

.68 

88.227 

5385 

-.587 

-.177 

68.23 

99.75 

5.858 

•620 

.081 

69.44 

99.81 

4383 

-.420 

-.190 

7539 

99.20 

4.606 

-.346 

-.435 

75.98 

99.21 

5323 

-.417 

-.075 

66.44 

99.69 

4,918 

-334 

-.025 

72.75 

99.05 

6332 

-.373 

.240 

59.27 

99.65 

5.213 

-.613 

-.087 

7133 

99.33 

4.831 

-371 

.173 

71.70 

100.00 

5.057 

-.440 

-.261 

73.41 

99.73 

5.004 

-.458 

-.088 

72.94 

98.79 

Tabic 


Correlations  Among  OS  Comrosnes  and  Factors  -  Form  A 


Comp 

Fad 

••JCl 

Suml* 

FSG 

rSG2'=' 

Writ" 

Perf- 

p/w« 

Suml 

.99 

FSG 

.97 

.96 

FSG2 

.98 

.98 

.99 

Writ 

.75 

.75 

.79 

jfi 

Perf 

.86 

.89 

.89 

.92 

il 

PW 

.92 

.88 

.87 

.86 

.61 

Phi 

.95 

.97 

.94 

.95 

.65 

.88 

.88 

Ph2^ 

."S 

.77 

.82 

.83 

.79 

.75 

.60 

Note,  n  =  109 

®Suml:  Average  of  tests  Security.  Adv  Man  Bdl.  Int  Comm/3Min.  Pubs/Logs,  Signal  Bk,  Ext  Comm.  Charts.  Coast  Nav.  Rules  Rd. 
Air/Surf  Tracker 

^FSG2:  Final  School  Grade  using  initial  test  scores 

*^Writ:  Average  of  tests  Sccur.  Basic  Man  Bd.  Int  Comm/3  Min.  Pubs/Logs.  Ext  Comm..  Air  Plot.  Charts.  Rules  Rd„  Air/Surf 
Tracker  -P.  Air/Surface  Tracker- W 

^Pcrf:  Average  of  Adv  Man  Bdl.  Adv  Man  Bd2.  Signal  Bk.,  Cart  Grid..  Surf  Summ..  Dead  Rcc..  Coast  Nav..  Radar  Scope  Op, 
.Air/Surf  Tracker  -  P 

®P,A\':  Average  of  Basic  Man  Bd,  Int.  Comm/3M,  Pubs/Logs.  Ext.  Comm..  Air  Plot 

-Phl:  Average  of  tests  Security  -  Air  Plot 

®Ph2:  Average  of  tests  Dead  Rcc  -  Air/Surf  Trackcr-P 

Table  86 

Correlations  Among  OS  Composites  and  Factors  -  Form  B 


Comp 

Fad 

Fac2 

Suml* 

Sum2^ 

FSG 

FSG2'^ 

Comm^ 

Plot* 

Writ^ 

Pcrf« 

P/W*^ 

Fac2 

.21 

Suml 

.90 

52 

Sum2 

.53 

.86 

.80 

FSG 

.78 

.73 

.92 

.89 

FSG2 

.79 

.74 

.93 

.91 

.99 

Comm 

.93 

.46 

.96 

.74 

.90 

.90 

Plot 

.57 

.84 

.80 

.95 

.91 

.93 

.73 

Writ 

.92 

.29 

.86 

-52 

.77 

.78 

.87 

.59 

Perf 

.56 

.82 

.81 

.93 

.91 

.92 

.74 

.93 

51 

P/W 

.63 

.65 

.72 

.76 

.80 

.81 

.72 

.75 

52 

.62 

Phi 

.69 

.79 

.87 

.91 

.96 

.95 

.82 

.91 

.66 

.88 

.85 

Ph2^ 

.78 

51 

.84 

.70 

.85 

.86 

.83 

.75 

.81 

.80 

.56 

Note,  n  =  471 

*Suml:  Average  of  tests  Security,  Adv  Man  Bdl.  Int  Comm/3Min,  Pubs/Logs,  Signal  Bk,  Ext  Comm,  Charts,  Coast  Nav,  Rules  Rd, 
Air/Surf  Tracker 

^Sum2:  Average  of  tests  Basic  Man.  Bd.  Adv.  Man  Bdl,  Signal  Bk,  Cart  Grid.  Surf  Summ,  Air  Plot,  Dear  Rec,  Coast  Nav. 

^FSG2:  Final  School  Grade  using  initial  test  scores 

^Comm:  Average  of  all  tests  associated  with  Communication  (Security,  Int.  Comm/3M,  Pubs/Logs,  Signal  Bk.,  Ext.  Comm.,  Charts 
^Plot:  Average  of  all  tests  associated  with  Plotting  (Cart.  Grid.  Surf  Summ,  Air  Plot,  Dead  Rec,  Coast  Nav.  Rules  Rd,  Radar  Scope 
Op,  Basic  Man  Bd,  Adv.  Man  Bdl,  Adv.  Man  Bd2 

^Writ:  Average  of  tests  Secur,  Basic  Man  Bd,  Int  Comm/3  Min,  Pubs/Logs,  Ext  Comm.,  Air  Plot,  Charts,  Rules  Rd.,  Air/Surf 
Tracker  -P,  Air/Surfacc  Trackcr-W 

^Perf:  Average  of  Adv  Man  Bdl,  Adv  Man  Bd2,  Signal  Bk.,  Cart  Grid.,  Surf  Summ.,  Dead  Rec..  Coast  Nav..  Radar  Scope  Op, 
Air/Surf  Tracker  -  P 

^Perf/Writ:  Average  of  Basic  Man  Bd,  Int.  Comm/3M,  Pubs/Logs,  Ext.  Comm.,  Air  Plot 

^Phasel:  Average  of  tests  Security  -  Air  Plot 

jphase2:  Average  of  tests  Dead  Rcc  -  Air/Surf  Trackcr-P 
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The  strongest  correlation  with  Factorl  of  Form  A  occurs  with  Suml  (.99), 
followed  closely  by  FSG  (.97).  Suml  and  FSG  also  correlate  highly  (.96). 

The  strongest  correlation  with  Factorl  of  Form  B  occurs  with 
Communication  (.93),  followed  closely  by  Written  (.92)  and  Suml  (.90).  Suml 
also  correlates  highly  with  Communication  (.96).  The  strongest  correlation  for 
Factor2  of  Form  B  occurs  with  Sum2  (.86),  followed  closely  by  Navigation/ 
Plotting  (.84)  and  Performance  (.82).  Sum2  correlated  highly  with  both 
Navigation/Plotting  (.95)  and  Performance  (.93). 

Recommendation.  Use  FSG  for  Form  A  and  Form  B.  For  Form  B  only, 
use  the  average  of  all  "Communication”  tests  and  the  average  of  all 
"Navigation/Plotting"  tests. 


also 


Radioman  (RM) 

Description  of  Variables.  The  Radioman  (RM)  school  assesses  student 
performance  by  progress  tests  and  within-course  comprehensive  tests.  These 
include  criterion-referenced  knowledge  and  performance  tests.  The  RM  school 
consists  of  11  weeks  of  instruction.  Student  action  code  reflects  graduation 
status  or  why  a  student  was  dropped  (see  Appendix  B). 

All  test  scores  in  the  database  represent  initial  scores  only.  Knowledge 
tests  require  a  minimum  score  of  80  to  pass.  The  school  allows  students  up  to 
2  retests;  however,  the  school  assigns  a  score  of  80  for  all  successful  retests. 
Performance  tests  require  a  minimum  score  of  100  to  pass.  Comprehensive  tests 
require  a  minimum  score  of  75  to  pass.  The  school  allows  students  up  to  2 
retests  on  the  comprehensive  tests  and  then  assigns  a  score  of  75  for  all 
successful  retests.  Test  #003  (the  Ace  Lab)  in  the  former  curriculum  requires 
a  minimum  score  of  70  to  pass.  The  school  allows  students  up  to  2  retests  and 
then  assigns  a  score  of  70  for  successful  Ace  Lab  retests.  If  a  student  fails 
a  2nd  retest,  the  Academic  Review  Board  (ARB)  determines  the  student’s  status. 
RGI  entered  the  number  of  retests  for  each  test  module  into  the  database  in 
addition  to  the  initial  test  score. 

The  RM  school  implemented  a  new  course  curriculum  in  October  1990.  The 
curriculum  variable  indicates  which  curriculum  the  student  received.  Both  the 
old  and  the  new  curricula  have  3  phases,  and  each  phase  has  a  comprehensive  final 
test.  Each  test  module  has  3  versions. 

Table  87  identifies  the  tests  by  number  and  describes  their  content  for 
both  the  former  and  the  present  curricula.  The  school  implemented  several 
changes  from  the  old  curriculum  to  the  new  curriculum.  In  Phase  I,  the  typing 
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Tabic  h" 

RM  V'anabies  in  the  Former  ar.j  Prcsenr  C'jmcula 


\ar.  No  Dcscnption  (Name  -ispc  ot  lest”) 


Former  Curheuium 

Phase  I 

210  knowledge  test  covenne  Communication  Organization  (Communication  Organization  -  K) 

330  knowledge  test  covering  Secunu*  (Sccunt>'  -  K) 

440  knowledge  test  covenng  Message  Format  (Parts,  Components.  Elements)  (Message  Format  -  K) 

441  ijtov^'ledge  test  covenng  Basic  Message  Format  (Basic  Message  Format  -  K) 

442  End  of  Phase  I  Comprehensive  test  (Phase  i  Comprehensive) 

Phase  11 

550  knowledge  test  covenng  Radiowavc  Propagation  and  Modes  of  Operation  (Radiowave  Propagation  •  K) 

610  knowledge  test  covenng  Safew  (Safety  -  K) 

510  knowledge  test  covering  Basic  Communications  Equipment  (Basic  Communications  Equipment  •  K) 

760  performance  test  covering  Communication  System  Construction  (Communication  System  Construction  -  P) 

511  performance  test  covenng  Planned  Maintenance  Sub-system  (PMS)  (Planned  Maintenance  Sub-system  -  P) 

025  knowledge  test  covenng  Theater  Nuclear  Warfare  (Theater  Nuclear  Warfare  -  K) 

133  end  of  Phase  II  Comprehensive  test  (Phase  II  Comprehensive) 

Phase  111 

110  penormance  test  covenng  Message  Processing  Inrouter/Filc  Clerk  (Message  Processing  Inrouter  -  P) 

131  knowledge  test  covenng  NAV'^MACS  Initialization  (.N’AVMACS  Initialization  -  K) 

132  penormance  test  covenng  NAV'MACS  BCST  Operation  (NAVMACS  BCST  Operation  -  P) 

S30  knowledge  test  covenng  .Message  Processing  Outrouter  (Message  Processing  Outrouter  -  K) 

930  knowledge  test  covenng  RTH'  Circuit  Operations.  Call  Signs,  Prosigns,  and  Operating  Signals  (RTTY  Circuit  Operations  -  K) 

123  knowledge  test  covenng  Circuit  Operations  and  Radiotelephone  (Circuit  Operations  -  K) 

(X)2  performance  test  covenng  Circuit  Operations  (  Circuit  Operations  -  P) 

464  knowledge  test  covering  Tapecutting  Corrections  (Tapccuitmg  Corrections  -  K) 

470  performance  test  co\*cnng  CGC-6  Operator  Maintenance  (UGC-6  Operator  Maintenance  -  P) 

026  end  of  Phase  HI  Comprehensive  test  (Phase  HI  Comprehensive) 

003  Ace  Lab  (Ace  Lab  -  P) 

Present  CZurtkuJuni 

Phase  ! 

210  knowledge  test  covering  Communication  Organization  (Communication  Organization  -  K) 

330  knowledge  test  covering  Security  (Security  -  K) 

441  knowledge  test  covering  Basic  Message  Format  (Basic  Message  Format  -  K) 

442  End  of  Phase  1  Comprehensive  test  (Phase  1  Comprehensive  -  K) 

Phase  II 

550  knowledge  lest  covering  Radiow-avc  Propagation  and  Modes  of  Operation  (Radiowavc  Propagation  -  K) 

610  knowledge  test  covering  Safety  (Safety  -  K) 

510  knowledge  test  covering  Basic  Communications  Equipment  (Basic  Communications  Equipment  •  K) 

760  performance  test  cov’cring  Communication  System  Construction  (Communication  System  Construction  -  P) 

511  performance  test  covering  Planned  Maintenance  Sub-system  (PMS)  (Planned  Maintenance  Sub-system  -  P) 

025  knowledge  test  covering  Theater  Nuclear  Warfare  (Theater  Nuclear  Warfare  -  K) 

133  end  of  Phase  H  Comprehensive  test  (Phase  II  Comprehensive) 

Phase  111 

110  performance  test  covering  Message  Processing  Inrouter/Filc  Qerk  (Message  Processing  Inrouter  -  P) 

131  knowledge  test  covenng  NAVMACS  Initialization  (NAVMACS  Initialization  -  K) 

830  knowledge  test  covering  Message  Processing  Outrouter  (Message  Processing  Outrouter  -  K) 

930  knowledge  test  covering  RTTY  Circuit  Operations,  Call  Signs,  Prosigns,  and  Operating  Signals  (RTTY  Circuit  Operations  -  K) 

123  knowledge  test  covering  Circuit  Operations  and  Radiotelephone  (Circuit  Operations  •  K) 

002  performance  test  covering  Circuit  Operations  (Circuit  Operations  -  P) 

464  knowledge  test  covering  Tapecutting  Corrections  (Tapecutting  Corrections  -  K) 

470  performance  test  covering  UGC-6  Operator  Maintenance  (UGC-6  Operator  Maintenance  -  P) 

026  end  of  Phase  HI  Comprehensive  test  (Phase  HI  Comprehensive) 


*Typc  of  Test:  K  -  Knowledge,  P  -  Performance 


qualification  changed  from  900  points  to  800.  The  school  deleted  Module  #440. 
(It  consisted  of  28  elements  of  basic  format  messages  testing  memorization.) 

In  Phase  II,  the  school  shortened  Module  #550  by  deleting  most  of  the  modes 
of  operation  and  moving  a  portion  of  it  to  module  #510.  The  school  also 
shortened  the  radio  wave  propagation  and  deleted  equipment  nomenclature.  The 
school  currently  includes  instruction  on  an  additional  piece  of  equipment  in 
Module  #510.  In  Phase  III,  the  school  deleted  module  #132  (a  performance  test 
of  broadcast  operations). 

In  both  old  and  new  curricula,  the  school  computes  Final  School  Grade  (FSG) 
as  the  average  of  all  knowledge,  performance,  and  comprehensive  tests. 

Sample.  Our  sample  included  only  6  students  who  completed  the  former 
curriculum.  Thus,  we  did  not  conduct  any  analyses  on  the  former  curriculum.  The 
remaining  293  students  in  our  sample  completed  the  current  curriculum. 

Results  and  Discussion.  For  the  current  curriculum,  the  retest  variables 
showed  very  low  variability,  with  between  0  and  73  retests  (median  3.5)  out  of 
291  cases;  thus,  we  did  not  analyze  any  retest  variables.  Tests  511,  464,  and 
470  had  standard  deviations  of  zero,  and  Test  002  had  a  standard  deviation  of 
only  .604,  so  we  dropped  these  4  variables  from  the  analysis.  Table  88  presents 
descriptive  statistics  on  the  variables  which  remained  for  the  factor  analysis. 

The  factor  analysis  indicated  two  factors.  Table  89  shows  the 
commonalities,  factor  loadings,  and  factor  score  coefficients.  The  tests  from 
Phase  I  and  Phase  II  tend  to  load  highest  on  Factorl,  while  tests  from  Phase  III 
load  higher  on  Factor2.  To  see  how  well  simple  composites  could  represent  these 
factors,  we  created  several  composites,  including  Phasel  (the  average  of  all 
tests  in  Phase  I),  PhaseZ  (the  average  of  all  tests  in  Phase  II),  and  Phase3  (the 
average  of  all  tests  in  Phase  III).  We  computed  PhaselZ  as  the  average  of  Phasel 
and  Phase2  to  represent  Factorl,  while  Phase3  represented  Factor2.  We  also 
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I'abic 

Descnrnve  Siatisiics  tor  RM  Vanabics 


\'anable 

jn 

Mean 

S:d  Dev 

Skew 

Kurtosis 

Minimum  Maximum 

Communication  Organization  -  K 

292 

S6.041 

11.267 

•1.042 

1.223 

-W.OOO 

100.000 

Sccunty  -  K 

291 

93.436 

6.117 

■1.089 

lil5 

65.000 

100.000 

Basic  Message  Format  -  K 

291 

88.804 

9.405 

•1.126 

1.915 

44.000 

100.000 

Phase  I  Comprehensive 

291 

92.357 

0.642 

-.643 

-.315 

68.000 

100.000 

Radiowave  Propagation  -  K 

291 

90.955 

9.278 

-1.351 

2.140 

52.000 

100.000 

Safety'  •  K 

291 

99.541 

2.305 

-6.110 

41.978 

80.000 

100.000 

Basic  Communications  Equipment  -  K 

291 

87.835 

9.335 

-.919 

1.242 

52.000 

100.000 

Communication  Sv’stcm  Construction  - 

P  291 

99.705 

2.185 

-9.761 

109.436 

71.400 

100.000 

Theater  Nuclear  Warfare  -  K 

291 

95i38 

6.044 

-1-533 

2.470 

66.700 

100.000 

Phase  il  Comprehensive 

291 

88.720 

6.986 

-.179 

-.749 

65.000 

100.000 

Message  Processing  Inrouier  -  P 

291 

98.287 

4.317 

-5.910 

47.898 

58.800 

100.000 

NAVM.ACS  Initialization  -  K 

291 

90.892 

"290 

-1.101 

1.839 

55.400 

100.000 

Message  Processing  Ouiroutcr  -  K 

291 

91.942 

■*.:’69 

-2.193 

10.038 

35.000 

100.000 

Rl  1 V  Circuit  Operations  -  K 

291 

99.189 

1.670 

-2.486 

6.813 

90.000 

100,000 

Circuit  Operations  -  K 

291 

93.634 

6.396 

-2.378 

10.724 

51.600 

100.000 

Phase  III  Comprehensive 

291 

91.005 

".300 

-  795 

.103 

65.600 

100.000 

Tabic  89 

Communalities.  Factor  Lx)adinES.  and  Factor  Score  Coefficienfs  for  RM  Variables 


Factor  Loadings _  Factor  Score  Coefficients 


Variable 

Communality 

Factor  1: 

Phase  I  &  II 

Factor  2; 

Phase  III 

Factor  1: 

Phase  I  &  II 

Factor  2: 
Phase  III 

Communication  Organization  -  K 

.350 

.470 

.359 

.120 

.085 

Secuniy  -  K 

.343 

.524 

.263 

.164 

.021 

Basic  Message  Format  -  K 

.387 

.600 

.166 

.259 

-.085 

Phase  I  Comprehensive 

.273 

.431 

.296 

.102 

.051 

Radiowave  Propagation  -  K 

.266 

.459 

.234 

.126 

.017 

Safely  -  K 

.015 

-.009 

.120 

-.024 

.054 

Basic  Communications  Equipment  -  K 

.:mo 

.527 

.249 

.181 

-.006 

Communication  System  Construa.  -  P 

.143 

.327 

-.190 

.161 

-.157 

Theater  Nuclear  Warfare  -  K 

.062 

.202 

.147 

.039 

»  .017 

Phase  II  Comprehensive 

.388 

.483 

.394 

.122 

.114 

Message  Processing  Inrouter  -  P 

.048 

.W2 

.215 

-.027 

.061 

NAVMACS  Initialization  -  K 

.402 

.171 

.610 

-.017 

J58 

Message  Processing  Outroutcr  -  K 

.255 

.348 

.367 

.058 

.109 

RTTY  Circuit  Operations  -  K 

.149 

.378 

-.074 

.163 

-.123 

Circuit  Operations  -  K 

.326 

.204 

.533 

-.054 

.257 

Phase  III  Comprehensive 

.326 

.370 

.435 

.059 

.132 

computed  FSG2  (the  simple  average  of  all  test  scores),  Know  (the  simple  average 
of  all  knowledge  tests),  and  Perf  (the  simple  average  of  all  Performance  tests). 

Table  90  displays  descriptive  statistics  for  these  composite  variables. 
Next,  we  computed  Pearson  correlation  coefficients  for  the  composite  variables. 
Table  91  presents  the  composite  variable  intercorrelations.  Since  FSG  and  FSG2 
correlate  .99,  this  offers  no  support  for  using  initial  test  scores  to  compute 
FSG. 

Phasel2  correlates  highest  (.92)  with  Factorl,  followed  closely  by  Know 
(.86),  Phasel  (.85),  and  FSG  (.84).  Similarly,  Phases  correlates  highest  (.87) 
with  Factor2,  followed  by  FSG  and  Know  (each  .73).  FSG  correlates  .93  with 
Phasel2  and  .80  with  Phases.  These  findings  support  using  FSG  as  an  indicator 
of  performance  on  Phases  1  and  2  but  suggest  using  a  separate  indicator  for 
performance  on  Phase  3. 


Table  90 

Descnptive  Statistics  for  RNI  Composiie  V^ariables 


Composite 

Rel“ 

.Mean 

Std  Dev 

Skew 

Kurtosis 

Minimum  Maximum 

FSG 

N/A 

94.628 

2.586 

-.074 

-.682 

87300 

99.600 

FSG2‘' 

.80 

93.005 

3JU 

-.213 

-340 

83340 

99330 

Know^ 

.81 

92.148 

3.930 

-.185 

-.610 

82.030 

99.640 

Perf* 

.08 

99.008 

2.398 

-5.046 

33.826 

79.400  . 

100.000 

Phasel* 

.68 

90.187 

6.034 

-.425 

-.337 

70.880 

100.000 

Phase2^ 

S2 

93.707 

3.654 

-.394 

-.363 

82.900 

100.000 

Pha$cl2* 

.75 

91.947 

4.348 

-.264 

-316 

79.140 

99.790 

Phasea** 

.60 

94.182 

3347 

-376 

-.303 

83.920 

100.000 

Note,  n  *  290. 

^These  are  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^FSG2:  FSG  computed  using  initial  test  scores. 

^Know:  Average  of  all  knowledge  tests. 

^Perf:  Average  of  the  2  performance  tests. 

®Phascl:  Average  of  the  4  Phase  1  tests. 

^Phase2:  Average  of  the  6  Phase  2  tests. 

*Phascl2:  (Phasel  +  Phase2)/2 
^Phasc3:  Average  of  the  6  Phase  3  tests. 
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Tdble 

Correiations  Among  RM  Composites  and  Factors 


Comp~ 

Fad 

FSG 

r‘SG2 

Know 

Perf 

Phase  1 

Phase2  Phasei2 

Fac2 

.29 

FSG 

.S4 

.73 

FSG2° 

.S6 

.73 

.99 

Know" 

.S6 

.73 

.98 

1.00 

Perf^ 

.23 

.14 

.29 

.28 

.20 

Phase  1® 

.85 

.49 

.84 

.86 

.86 

.16 

Phase2^ 

.80 

.47 

.82 

.84 

.83 

.24 

.59 

Phase 12« 

.92 

.54 

.93 

.95 

.95 

.21 

.94 

.S3 

Phases^* 

.49 

.87 

.80 

.80 

.79 

.31 

.53 

00 

Note,  n  =  290. 

^Thesc  arc  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^FSG2:  FSG  computed  using  initial  test  scores. 

^Know-  Average  of  all  knowledge  tests. 

^Perf:  Average  of  the  2  performance  tests. 

®Phascl:  Average  of  the  4  Phase  1  tests. 

*Phasc2:  Average  of  the  6  Phase  2  tests. 

*Phasei2:  fPhasel  +  Phasc2)/2 
*'‘‘Phase3;  Average  of  the  6  Phase  3  tests. 


Recommendations.  In  addition  to  FSG,  use  Phases  (an  average  of  the  tests 


in  Phase  3) . 


Army  Schools 

Field  Artillery  Fire  Support  Specialist  (13F) 

Description  of  Variables.  The  13F  school  has  three  areas  of  training: 
communications,  map  reading/land  navigation,  and  fire  support.  The  school 
generates  composite  scores  for  the  instructional  departments  of  Communications 
and  Electronics  (Test  1),  Target  Acquisition  (Tests  2-4),  and  Fire  Support  and 
Combined  Arms  Operations  (Tests  5-13).  In  addition,  the  school  generates 
composite  scores  for  branches  within  each  department.  The  composite  scores  for 
the  Communications  and  Electronics  and  Target  Acquisition  Departments  equal  their 
respective  branch  scores  since  each  department  has  only  one  branch.  The  Fire 
Su-port  and  Combined  Arms  Operations  Department  branch  composites  consist  of  13F 
duties  and  knowledge  of  the  Fire  Support  Vehicle  (FSV)  (Tests  5  and  6),  advanced 
13F  duties  (Tests  7  and  8),  and  Target  Shoots  (9-11). 

The  school  calculates  composite  scores  by  dividing  the  number  of  points 
achieved  by  the  number  of  points  possible.  Since  researchers  can  generate  the 
composite  scores  from  the  test  scores,  we  did  not  enter  the  composite  scores  in 
the  database.  The  school  calculates  the  Final  School  Grade  (FSG)  by  taking  the 
sum  of  all  performance  and  written  test  scores  and  dividing  by  the  number  of 
points  possible  (1,000). 

The  school  assigns  a  score  equal  to  70%  of  the  test  maximum  to  all  retests. 
For  each  of  Tests  1-13,  the  number  of  tries  variables  indicate  the  number  of 
trials  needed  by  the  student  to  pass  each  test.  A  0  equals  test  not 
administered,  a  1  equals  passed  on  first  attempt,  and  a  2  equals  passed  on  second 
attempt.  The  researchers  recoded  scores  of  0  as  missing.  Each  time  the  number 
of  tries  equals  2,  the  score  will  have  a  value  equal  to  70%  of  the  maximum 
because  the  school  assigns  the  minimum  passing  score  to  all  those  passing  a  retest. 
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Students  receive  an  action  code  (see  Appendix  B)  only  if  dropped  from  the 
class.  The  school  provided  information  on  each  student’s  rank  and  the  number  of 
students  in  the  class,  each  of  which  could  range  from  1  to  40.  The  researchers 
computed  a  measure  of  class  standing  using  student  rank  (r)  and  class  size  (n): 
(class  size  -  rank)/class  size  =  (n-r)/n.  The  class  standing  variable  puts 
students  on  a  metric  of  0  to  approaching  1,  so  that  the  higher  the  standing,  the 
better  the  student  did  relative  to  the  other  members  of  the  class.  Further,  this 
measure  takes  into  account  class  size.  For  example,  a  person  who  ranked  first 
out  of  a  class  of  40  would  have  a  standing  of  .975,  while  a  person  who  ranked 
first  out  of  a  class  of  5  would  have  a  standing  of  .80. 

The  first  3  digits  of  the  class  number  indicate  the  number  of  the  class, 
the  last  2  digits  indicate  the  year  of  the  class  (90  or  91).  Sometimes 
instructors  divide  classes  into  sections  of  approximately  10  students,  indicated 
by  the  section  number  variable. 

Approximately  26%  of  the  trainees  took  part  in  a  fast  track  training 
program.  The  fast  track  students  had  higher  GT  scores  and  higher  initial  entry 
scores  than  did  the  regular  track  students.  The  fast  track  soldiers  received 
faster  training  which  covered  the  same  areas  as  that  of  the  regular  track,  but 
also  included  additional  training,  some  of  which  included  skill  level  2.  The 
school  gave  tests  8  and  13  to  fast  track  students  only.  Fast  track  students  did 
not  receive  tests  7  and  12.  Both  tests  8  and  13  contained  items  covering  the 
same  content  as  tests  7  and  12,  respectively,  but  also  contained  items  covering 
the  additional  training  the  fast  track  students  had  received.  Other  than  tests 
7  and  8  and  12  and  13,  both  tracks  received  identical  tests.  The  school  scores 
tests  12  and  13  as  pass/fail  and  does  not  include  them  in  composite  scores.  The 
variables  used  for  tests  12  and  13  indicate  the  number  of  trials  needed  to  pass 
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each  test.  The  school  dropped  the  fast  track  program  in  October  1991. 

Table  92  gives  the  specific  test  descriptions,  where  W  refers  to  a  written 
test  and  P  refers  to  a  performance  test,  and  the  numbers  in  parentheses  give  the 
valid  ranges  (from  passing  at  the  70%  level  to  the  maximum  possible).  For  each 
of  Tests  1  through  13,  we  also  recorded  number  of  tries  as  a  digit:  0  (recoded 
as  missing) ,  1,  or  2. 

The  school  uses  the  sum  of  two  map  reading/1  and  navigation  performance 
tests  (Tests  3  and  4)  twice:  once  as  the  score  for  Test  3  and  again  as  the  score 
for  Test  4.  We  excluded  Test  4  and  Trials  4  from  the  factor  analysis.  "All  of 
the  variables  before  the  Final  Test  are  commensurate  (i.e.,  the  same  exercises 
scored  the  same)  across  tracks"  (C.  B.  Walker,  personal  communication,  January 
23,  1992).  For  this  reason,  we  combined  Tests  7  and  8  into  a  single  variable. 
All  fast  track  students  passed  Test  13  (the  final  written  test)  on  the  first  try. 
For  this  reason,  the  researchers  combined  Tests  12  and  13  into  a  single  variable. 


Table  92 

Descriptions  of  13F  Variables 


Test 

Description 

Type* 

Passing  Range 

1 

Radio  and  Communication 

w 

143.50.205.00 

2 

Map  Symbols  and  Features 

w 

66^  95,00 

3 

Day  Land  Navigation  by  Terrain  Association 

p 

28.00. 40,00 

4 

Map  and  Terrain  Association  Field  Exercise 

p 

28.00.40.00 

5 

Duties  of  the  Fire  Suppon  Specialist 

w 

5^50.  75,00 

6 

Fire  Support  Vehicle  (FSV) 

p 

94J50.13S.00 

7 

Digital  Message  Device  (DMD)  &  Firing  Test  for  Regular  Track  Soldiers 

w 

77.00.110.00 

8 

Digital  Message  Device  (DMD)  &  Firing  Test  for  Fast  Track  Soldiers 

w 

77.00-110.00 

9 

Practice  Shoot  #1 

p 

70.00-100.00 

10 

Practice  Shoot  #2 

p 

70.00-100.00 

11 

Practice  Shoot  #3 

p 

70.00-100.00 

12 

Final  Test,  Regular  Track  (Pass/Fail),  number  of  tries 

w 

digits  0, 1,  or  2 

13 

Final  Test,  Fast  Track  (Pass/Fail),  number  of  tries 

w 

digits  0, 1,  or  2 

Note.  Each  of  Tests  1  through  11  also  had  a  variable  denoting  the  number  of  trials  to  pass  the  test,  where  0  indicated  test  not 
administered.  1  indicated  passed  on  first  attempt,  and  2  meant  passed  on  second  attempt. 

*W  *  Written,  P = Performance 


141 


Sample.  The  sample  consisted  of  755  regular  track  and  260  fast  track 
soldiers  (four  cases  had  unspecified  status).  The  researchers  categorized 
soldiers  into  track  status  on  the  basis  of  Tests  7  and  8,  since  the  data  did  not 
contain  an  explicit  track  indicator  variable.  Seventeen  trainees  who  had  scores 
on  (the  fast  track)  Test  8  also  had  scores  on  (the  regular  track)  Final  Exam  Test 
12  rather  than  (the  fast  track)  Test  13. 

The  test  forms  contained  three  cases  with  values  below  the  passing  level, 
one  each  on  Test  2,  Test  5,  and  Test  7.  Two  of  these  cases  had  missing  test 
data,  and  the  researchers  excluded  them  from  factor  analysis;  the  researchers 
retained  the  third  case.  The  researchers  excluded  from  analysis  two  cases  which 
had  entries  for  both  Test  7  and  Test  8.  We  excluded  from  further  analysis  the 
number  of  trials  for  Test  6  due  to  lack  of  variability.  The  final  sample 
consisted  of  1,006  cases:  750  regular  track  soldiers  and  256  fast  track 
soldiers. 

Results  and  Discussion.  We  combined  the  two  tracks  for  the  analysis,  since 
the  sample  consisted  of  only  26%  fast  track  soldiers  and  the  school  dropped  the 
fast  track  program  in  October  1991.  Table  93  displays  descriptive  statistics  for 
the  variables.  Because  Tests  7  and  8  had  the  same  timing  and  because  the 
observed  mean  scores  represent  a  conservative  estimate  of  the  difference  between 
the  regular  and  the  fast  track  students,  we  considered  it  appropriate  to  treat 
Tests  7  and  8  as  one  variable. 
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Table  93 

Descnpttve  Statistics  for  13F  Variables 


\‘ariablc  Mean  Std  Dev  Skew  Rurtosis  Minimum  Maximum 


1  (Radio  and  Communication) 

2  (Map  Sv-mbols  and  Features) 

3  (Day  Land  Navigation  by  Terrain  Association) 

4  (Map  and  Terrain  Association  Field  Exercise) 

5  (Duties  of  the  Fire  Suppon  Specialist) 

6  (Fire  Suppon  Vehicle) 

7  (Digital  Message  Device  &  Firing  Test.  Regular)  (n  =  750) 

8  (Digital  Message  Device  &  Firing  Test.  Fast)  (n-256) 

7  &  8  Combined  (r\=  1.006) 

9  (Practice  Shoot  #1) 

10  (Practice  Shoot  #2) 

11  (Practice  Shoot  #3) 

12  (Final  Test.  Regular)  (n  =  766) 

13  (Final  Test.  Fast)  (ji  =  240) 

12  &  13  Combined  (ji*  1.006) 

Class  Standing 


194.604 

12.033 

-1.622 

2.702 

143300 

205.000 

81.808 

8.681 

-.316 

-.872 

66300 

95.000 

33.153 

4.427 

.240 

-1.297 

28.000 

40.000 

33.153 

4.427 

,240 

-1.297 

28.000 

40.000 

69.463 

5.492 

-2.600 

20.948 

3.000 

75.000 

130.727 

6.906 

-2.077 

5.026 

94300 

135.000 

93.058 

9.431 

-.121 

-1.033 

77.000 

110.000 

95.335 

8i31 

-.401 

-.762 

77.000 

110.000 

93.638 

9.259 

-.199 

-.989 

77.000 

110.000 

88.341 

8.838 

-.583 

-.574 

70.000 

100.000 

88.013 

8.806 

-.544 

-.644 

70.000 

100.000 

89.888 

8.667 

-.774 

-.225 

70.000 

100.000 

1.085 

.279 

2.985 

6.930 

1.000 

2.000 

1.000 

.000 

- 

— 

1.000 

1.000 

1.065 

.246 

3347 

10.605 

1.000 

2.000 

.476 

.285 

.005 

-1,193 

.000 

.974 

Note.  Total  n  =  i.uC*6.  16  students  who  had  taken  Test  8  (Fast  Track)  took  Test  12  (Regular  Track)  instead  of  Test  13. 


The  researchers  first  conducted  factor  analysis  on  the  set  of  all  test  and 
trials  variables  (except  Test  4,  trials  for  Test  4,  and  trials  for  Test  6).  The 
test  variables  correlated  in  the  low  range  with  one  another  (.048  to  .338), 
indicating  that  the  tests  measure  relatively  independent  aspects  of  the  Fire 
Support  Specialist  duties.  The  scree  test  indicated  two  factors.  However,  the 
trials  variables  and  the  final  exam  displayed  very  low  commonalities  (.037  to 
.220)  and  loadings  (.013  to  .457  in  absolute  value).  We  ran  the  factor  analysis 
again  without  the  trials  variables  and  without  the  final  exam,  and  we  found  the 
same  two  factors.  Thus,  we  present  these  results  based  on  analyzing  just  the 
tests. 

Table  94  shows  the  communal i ties,  factor  loadings,  and  factor  score 
coefficients.  The  primary  factor  consists  of  the  sum  of  Tests  1,  2,  3,  and  5. 
This  factor  resembles  a  "Map  and  Radio"  factor  due  to  the  tests  which  load  on  it. 
The  second  factor  contains  Tests  7  &  8,  9,  10,  and  11,  which  we  labeled  the 
"Firing"  factor. 
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Table  94 

Communalitics.  Factor  Loadings,  and  Factor  Score  Coefficienis  Cor  13F  V^ariahles 


Test 

Communality 

Factor  Loadings 

Factor  Score  Coefficients 

Factor  1: 

Map  &  Radio 

Factor  2: 

Firing 

Factor  1; 

Map  &  Radio 

Faaor  2: 

Firing 

1  (Radio  &  Communication) 

.310 

SIS 

.213 

.274 

.017 

2  (Map  SvTTibols  &  Features ) 

.396 

i98 

.197 

.382 

-.019 

3  (Day  Land  Navigation) 

.143 

.368 

.085 

.168 

-.028 

5  (Duties  of  the  FSS) 

.265 

.482 

.180 

.247 

.003 

6  Fire  Suppon  Vehicle) 

.066 

.108 

.232 

.009 

.099 

7&8  (Digital  Message  Dev.) 

.292 

.298 

.451 

.073 

.231 

9  (Practice  Shoot  #1) 

.254 

.140 

.484 

-.032 

.277 

10  (Practice  Shoot  #2) 

.256 

.149 

.483 

-.027 

.276 

11  (Practice  Shoot  #3) 

.233 

.102 

.472 

-.048 

.272 

Note,  n  =  1.006. 


Wg  sought  to  develop  composite  variables  which  would  capture  essential 
elements  of  the  criterion  variance  and  also  represent  meaningful  content 
dimensions  useful  to  the  13F  school.  Correlations  with  the  empirical  factors 
would  illustrate  the  degree  to  which  the  simpler  unit-weighted  composites  could 
represent  the  empirically  defined  factor.  Accordingly,  we  developed  the  Map  and 
Radio  composite  (the  sum  of  Tests  1,  2,  3,  and  5,  divided  by  415)  to  represent 
Factor  1,  and  the  Firing  composite  (the  sum  of  Tests  7  &  8,  9,  10,  and  11, 
divided  by  410)  to  represent  Factor  2.  The  school  uses  FSG,  sothe  researchers 
computed  FSG2  (using  initial  test  scores)  to  examine  the  pattern  of 
intercorrelations  with  FSG.  In  order  to  examine  whether  a  written  test  versus 
performance  test  distinction  could  explain  aspects  of  school  performance,  we 
calculated  the  sum  of  the  written  tests  (1,  2,  5,  and  7  &  8)  divided  by  485  and 
the  sum  of  the  performance  tests  (3,  4,  6,  9,  10,  and  11)  divided  by  515.  We 
examined  the  sum  of  the  final  three  tests  (Tests  9,  10,  and  11,  the  artillery 
shoots),  because  earlier  research  (Kieckhaefer  &  Brantner,  1990)  had  indicated 
their  possible  role  in  describing  important  aspects  of  13F  duties.  Further  the 
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school  uses  Communications  and  Electronics  (Test  1),  Target  Acquisition  (the  sum 
of  Tests  2,  3,  and  4),  and  Fire  Support  and  Combined  Arms  Operations  (the  sum  of 
Tests  5  through  11)  as  composite  scores.  We  placed  all  the  composite  variables 
(except  class  standing  and  the  trials  variable)  on  a  0-100  scale  in  order  to 
compare  them  more  readily. 

Table  95  lists  descriptive  statistics  on  various  composite  variables,  while 
Table  96  lists  their  intercorrelations  and  their  correlations  with  the  factors. 
Since  FSG2  correlated  nearly  perfectly  with  FSG,  this  offers  no  support  for 
computing  FSG  using  initial  test  scores. 

As  indicated  in  Table  96,  the  Map  and  Radio  composite  correlated  the 
h-ghest  (.98)  with  Factor  1,  and  the  Firing  composite  correlated  highest  (.99) 
with  Factor  2.  These  strong  correlations  and  the  relatively  low  intercorrelation 
between  them  (r  =  .38)  make  the  Map  i  Radio  and  the  Firing  composites’ 

comprehensive  and  useful  summary  measures  which  at  the  same  time  measure 
different  aspects  of  the  Fire  Support  Specialist  school  performance. 

FSG  correlates  less  strongly  with  Factor  1  and  Factor  2  (.79  and  .82, 
respectively)  and  with  Map  and  Radio  and  Firing  (.81  and  .82).  This  suggests  FSG 
assesses  both  these  areas. 

Recommendations.  In  addition  to  FSG,  use  the  Map  &  Radio  and  Firing 
composite  scores. 
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Table  V5 

Descnruve  Statistics  tor  13F  Composne  Variables 


Composite 

Rel^ 

Mean 

Std  Dev 

Skew 

Kurtosis 

Min 

Max 

Map  &  Radio^ 

.60 

91.308 

3.346 

-.829 

383 

72347 

100.000 

Firing* 

-58 

87.767 

3.677 

-.403 

-.339 

70376 

100.000 

FSG 

S/A 

90.269 

4.113 

-307 

.006 

75.700 

99.300 

FSG2‘^ 

.72 

90.261 

4.130 

-312 

-.004 

75.710 

99.300 

Writ* 

.63 

90-598 

3342 

-.727 

.339 

72.907 

100.000 

Perf^ 

i8 

89.945 

3.323 

-.428 

-.262 

75.825 

100.000 

Shoot® 

.50 

88.742 

3.389 

-349 

-.216 

70.000 

100.000 

CE^ 

.31 

94.902 

2.678 

-1.630 

2.742 

70.000 

100.000 

Target^ 

.62 

S4.622 

3.369 

-.047 

-.891 

70.000 

100.000 

Supp- 

.60 

90.319 

3327 

-316 

-.142 

76.913 

99.495 

Stand^ 

N/A 

.476 

.285 

.005 

-1.193 

0.000 

0.970 

Trials^ 

N'/A 

1.CM8 

.083 

1.820 

3.122 

1.000 

1350 

Note,  n  =  1,006. 

^These  arc  composites  based  on  communalirv*  estimates  and  the  procedures  described  in  the  text, 
^Map  &  Radio;  lOOfSum  of  Tests  1.  2.  3,  5)/4l5 
'^Firing:  lOOfSum  of  Tests  7&S.  9,  10.  U)/410 
"FSG2:  FSG  based  on  initial  test  scores. 

*Wrii:  lOOfSum  of  Written  Tests )/485 

“Perf:  iOOfSum  of  Performance  Tests  )/5 15 

*Shoot;  lOOfSum  of  Artillerv'  Shoots)/300 

-CE:  lOOCTcsi  l)/205 

^Target:  lOOfSum  of  Tests  2-4)/175 

•^Supp:  lOOfSum  of  Tests  5-ll)/620 

^Stand:  Class  Standings  (class  size  -  rank)/class  size 

•Trials:  Mean  number  of  attempts  on  Tests  1-13 
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Tabic  96 

Correlations  Among  13F  Comc>osites  and  Factors 


Note,  n  *  1.006. 

®Thcse  are  composites  based  on  the  procedures  described  in  the  text. 
^.Map  Radio:  100(Sum  of  Tests  I.  2,  3.  5)/415 
^Firing:  lOOfSum  of  Tests  7&8.  9.  10.  11)/410 
^FSG2:  FSG  based  on  initial  test  scores. 

®Writ:  lOOrSum  of  Written  Tests)/485 
^Perf:  lOOfSum  of  Performance  Tests)/515 
8Shoot:  lOOfSum  of  Anillcry  Shoots )/300 
^CE:  100(Test  l)/205 
^Target:  lOOCSum  of  Tests  l^AyilS 


•^Supp:  lOOfSum  of  Tests  5-11V620 

^Stand:  Class  Standing '(class  size  -  ranlc)/class  size 

^Trials:  Mean  number  of  attempts  on  Tests  1-13 
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Heavy  Anti  armor  Weapons  Crewman  (IIH) 

Descriptions  of  Variables.  Students  in  the  IIH  school  receive  the  same 
training  and  testing  up  until  the  point  when  they  split  up  and  attend  either 
HMMWV  or  ITV  training.  First,  the  IIH  school  tests  all  students  on  8  TOW 
Performance  Training  Objectives  (TOs).  Then  the  school  tests  all  students  in  TOW 
Tracking  Performance,  consisting  of  up  to  8  Events.  After  completing  TOW  2 
system  training,  students  attend  either  HMMWV  or  ITV  training  school.  Each  ITV 
training  objective  consists  of  several  performance  measures,  and  most  of  the 
HMMWV  TOs  consist  of  several  performance  measures.  The  school  then  tests 
students  on  TOW  simulator  tracking  measures  for  HMMWV  and  ITV  schools.  Students 
pp'-form  up  to  3  Events  for  HMMWV  and  up  to  5  Events  for  ITV. 

We  distinguished  between  students  not  tested  on  a  performance  measure  and 
students  with  missing  values  on  a  performance  measure.  The  numbers  documented, 
for  each  performance  measure  represent  the  number  of  NO  60s  for  that  performance 
measure.  For  students  who  did  not  complete  a  task  within  the  time  allowed,  the 
instructor  documented  subsequent  performance  measures  with  a  NO  60. 

The  Student  Action  Code  reflects  why  the  IIH  school  dropped  a  student  and 
has  the  following  categories:  l=Medical  Reasons,  2=Failed  to  Qualify, 
3=Discharged,  and  4=Unknown.  The  data  contained  no  information  on  student  action 
code. 

TOW  Performance  TOs:  The  8  TOW  training  objectives  and  performance 
measures  consist  of  those  shown  in  Table  97.  The  school  instructors  administered 
the  several  performance  measures  shown  for  each  training  objective  as  60/N0  60 
performance  tests. 
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Table  97 

Performance  Measures  for  IIH  TOW  Training  Objectiy^ 


Training  Objectiv'e 

Performance  Measures 

1:  Assemble 

TOW  Launcher 

1.  Place  tnpod  legs  in  locked  position 

2.  Level  tripod 

3.  Carry  traversing  unit  by  control  knobs 

4.  Lock  groove  coupling  clamp 

5.  Carry  daysight  tracker 

6.  Lock  daysight  tracker 

7.  Secure  launch  tube 

8.  Handle  nightsight  properly 

9.  Check  nightsight 

10.  Set  nightsight  FOV  switch 

11.  Lock  nightsight  handle 

12.  Connect  post  amplifier  cable 

13.  Connea  battery 

14.  Install  battery 

15.  Connect  ceil  cord 

16.  Name  each  component. 

2:  Inspect  TOW  2 
Launcher  and  Missile 

1.  Conduct  sv-stem  check-out 

2.  Report  malfunctions 

3.  Perform  preoperational  missile  inspection. 

3:  Maintain  TOW  2 
Launcher 

1.  Conduct  visual  inspection 

2.  Describe  correction  of  deficiencies 

3.  Name  cleaning  material 

4.  Repon  any  deficiencies 

5.  Describe  how  to  spot  paint  the  launcher 

6.  Check  desicant  bags 

7.  Check  humidity  indicators. 

4:  Load  Encased 

Missile 

1.  Lock  down  system 

2.  Raise  bridge  clamp  and  close  trigger  cover 

3.  Inspect  missile 

4.  Remove  electrical  connector  dust  cover 

5.  Handle  and  load  missile  correctly 

6.  Use  two  hands  to  close  and  lock  bridge  clamp 

7.  Check  backblast  area 

8.  Raise  arming  lever 

9.  Lower  arming  lever 

10.  Unload  missile  correctly 

11.  Replace  forward  handling  rings 

12.  Tag  missile. 

5:  Action  for 

TOW  2  Misfire 

L  Check  battery,  coil  cord  connector  &  bridge 
clamp 

2.  Lower  and  raise  arming  lever 

3.  Perform  correct  actions  after  third  misfire 

4.  Remain  dear  of  front  and  rear  of  launcher. 

6:  Determine  if 

Target  is  Engageable 

1.  Specify  if  target  1  is  within  range,  engageable. 
or  not  engageable. 

(Performance  measures  2  through  5  follow 
similarly  for  targets  2  through  5.) 

7:  Prepare  Aniiarmor 
Range  Card 

la.  Correct  weapon  symbol 

lb.  Arrows  drawn  to  weapons  position 

lc.  Left  and  right  limits 

ld.  All  target  engagement  locations 

le.  TRPs,  to  include  the  TRP  number 

lf.  A  maximum  engagement  line 

lg.  All  prominent  terrain  features 
lb.  All  dead  space 

li.  Unit,  not  to  exceed  company 

lj.  Magnetic  north  arrow 

lk.  Type  of  position 

ll.  Type  of  weapon 

lm.  Correa  interval  between  circles 

ln.  Date 

lo(l).  Dtrcaion  on  target  engagement  locations 
lo(2).  Range  on  target  engagement  locations 
lo(3).  Description  on  target  engagement  locations 
lo(4).  TRP  number  for  all  TRPs  on  target 
engagement  location 

2.  Ensure  range  card  is  readable 

3.  Sute  two  copies  required. 

8:  Determine  TOW 
Firing  Limitations 

1.  Reduction  of  range  over  bodies  of  water 

2.  Fire  from  TOW  carriers 

3.  Fire  over  electrical  wires 

4.  Fire  in  windy  conditions 

5.  Fire  through  smoke  and  area  fires 

6.  Fire  from  bunkers  and  building? 

6a.  Room  size 

6b.  Ventilation 

6c.  Debris 

6d.  Noise 

6e.  Qeaiance  requirements 

6f.  Muzzle  clearance. 
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TOW  Tracking  Performance  Events;  The  school  uses  a  TOW  Tracking 
Performance  Scorecard  to  document  students’  TOW  tracking  scores.  Students  may 
perform  in  up  to  8  Events.  The  school  defines  an  Event  as  the  sum  of  10  shots. 
The  score  for  each  shot  (0-100)  reflects  time  on  target  from  firing  to  hit/miss. 
We  entered  Event  totals  as  they  appeared  on  the  scorecard.  Researchers  who 
calculate  Event  totals  may  find  different  Event  totals  than  those  reported  by  the 
school  due  to  arithmetic  errors  by  school  instructors.  The  school  defines  the 
Qualification  Score  as  the  highest  Event  total.  School  instructors  state  that 
students  frequently  practiced  more  than  what  instructors  documented  on  the 
scorecard.  Each  of  Events  1  through  8  consist  of  10  Event  shots  and  a  total 
score.  The  Qualification  Score  can  range  from  O-lOQQ  and  has  the  following 
categories:  0-549  Unqualified,  550-649  2ND  Class,  650-749  1ST  Class,  and  750- 
1000  Expert.  Thus,  the  school  sets  550  as  the  minimum  passing  score  for  the  TOW 
Tracking  Performance. 

The  IIH  school  defines  each  shot  as  identical  to  the  others,  with  the 
following  exception.  The  school  specifies  that  students  wear  protective  masks 
for  the  last  five  shots  in  practice  Event  2.  We  found  that  the  instructor 
sometimes  documents  these  scores  elsewhere  or  does  not  document  them  at  all. 

HMMWV  or  ITV  Training  School:  After  completing  TOW  2  system  training, 
students  attend  either  HMMWV  or  ITV  training  school.  The  training  indication 
variable  identifies  which  school  the  student  attended.  Most  often,  the 
qualification  score  for  HMMWV  or  ITV  training  reflects  the  sum  of  ten  shots  on 
the  student’s  final  Event.  However,  we  found  cases  where  the  qualification  score 
does  not  reflect  the  student’s  final  Event  score  or  the  highest  score. 

HMMWV  TOs;  The  HMMWV  TOs  consist  of  the  performance  measures  shown  below: 

1.  Stow  BII  on  HMMWV 

2.  Conduct  PMCS  on  HMMWV 
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3.  Install  and  Stow  the  M220A1 

4.  Destruction  of  the  HMMWV  ^ 

5.  Prepare  the  M966:  Squad  Leader;  Gunner;  Loader/Dnver 

6.  Load.  Arm,  Unload:  Gunner;  Loader/Driver 

7.  Reload:  Gunner:  Loader/Driver  _ 

8.  Immediate  Action:  Squad  Leader;  Gunner;  Loader;  D>"iver 

9.  React  to  Fire  Command:  Squad  Leader;  Gunner;  Loader/Dnver  . 

10.  Dismount  and  Assemble  the  M220A2:  Squad  Leader;  Gunner;  Loader/Dnver 

11.  Installing:  Squad  Leader;  Gunner;  Loader/Driver 

12.  Recognize  Friendly  and  Threat  Vehicles. 

ITV  TOs:  The  ITV  TOs  consist  of  the  performance  measures  shown  below: 

M243  Smoke  Grenade  Launcher:  Load;  Arm;  Fire;  Unload 
M60  Machine  gun:  Mount;  Stow;  Dismount 
Crew  Drill:  Squad  Leader;  Gunner;  Loader;  Driver 

Dismount  and  Assemble  the  M220A1:  Squad  Leader;  Gunner;  Loader;  Driver 
Intercom  Equipment:  Receive  Message;  Transmit  Message 
Turret  Operation:  Squad  Leader;  Gunner;  Loader;  Driver 
Troubleshooting:  Squad  Leader;  Gunner;  Loader;  Driver 
Reload  Dual  Launcher:  Gunner;  Loader 
Immediate  Action:  Squad  Leader;  Gunner;  Loader;  Driver 
React  to  Fire  Commands:  Squad  Leader;  Gunner 
Determine  Target  Engageabil ity:  Squad  Leader;  Gunner 

12.  Emergency  Action  Procedures:  Squad  Leader;  Gunner;  Loader 

13.  Manual  Acquisition  and  Tracking:  Squad  Leader;  Gunner;  Loader 

14.  Engage  Targets:  Gunner 


1. 

2. 

3. 

4. 

5. 

6. 
7. 

s'. 

10. 

11 


TOW  Simulator  Tracking  Events:  The  final  set  of  Event  data  represent  TOW 
simulator  tracking  scores  for  HMMWV  and  ITV  schools.  Students  perform  up  to  3 
Events  for  HMMWV  and  up  to  5  Events  for  ITV.  (To  determine  whether  these  data 
represent  HMMWV  or  ITV  Events  in  the  database,  researchers  should  refer  to  the 
Training  Indication  Variable.)  The  HMMWV  Events  consist  of  M966  TOW  Tracking 
Performance  Events.  The  ITV  Events  consist  of  ITV  Tracking  Performance  Events. 
We  entered  Event  totals  as  they  appeared  on  the  scorecard.  Again,  researchers 
who  calculate  Event  totals  may  find  different  Event  totals  than  documented  by  the 
school  due  to  instructors*  errors.  Finally,  school  instructors  stated  that 
students  frequently  practiced  more  than  what  instructors  documented  on  the 
scorecard.  Each  of  Events  1  through  5  consist  of  10  identical  Event  shots  (0- 
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100)  and  a  total  score  (0-1000).  The  Qualification  Score  represents  the  highest 
event  total,  with  the  following  classes:  0-599  Unqualified,  600-699  2ND  Class, 
700-799  1ST  Class,  and  800-1000  Expert.  Thus,  the  school  sets  the  minimum 
passing  score  for  the  HMMWV  and  ITV  Simulator  Tracking  Performance  Events  as  600. 

Sample.  The  ratio  scale  measures  in  the  IIH  school  consist  of  the  TOW 
Events,  with  shot  scores  that  can  range  from  0  to  100,  measuring  time  on  target 
from  firing  to  hit/miss.  These  occur  after  the  8  training  objectives  for  all 
students  and  also  at  the  end  of  training  for  ITV  and  HMMWV  students  separately. 
Since  the  two  groups  of  students  receive  different  training  and  different 
assessment  after  TOW  2  system  training,  it  makes  sense  to  analyze  the  two  groups 
separately.  The  quantitative  nature  of  the  Event  shot  data  make  them  amenable 
to  factor  analysis. 

The  ordinal  data  represent  the  number  of  NO  GOs  on  each  performance 
measure,  the  number  of  trials  to  pass  the  task.  The  Army  requires  that  all 
trainees  eventually  pass,  and  empirically,  almost  all  trainees  pass  on  the  first 
trial,  resulting  in  little  or  no  variability. 

The  researchers  recoded  the  number  of  NO  GOs  so  that  higher  scores 
represent  better  performance,  i.e.,  3  for  passing  on  the  initial  trial  (0  NO 
GOs);  2  for  passing  on  the  second  trial  (1  NO  GO);  and  1  for  passing  on  the  third 
trial  (2  NO  GOs).  These  item  level  data  still  resulted  in  limited  variability. 

The  researchers  then  grouped  item  level  data  into  meaningful  content 
categories  using  the  recoded  number  of  NO  GOs.  We  added  scores  within  a  TO  to 
define  a  summary  measure,  which  produced  8  TO  measures  for  the  first  part  of  the 
testing.  We  used  a  similar  approach  for  the  separate  ITV  or  HMMWV  training.  The 
ITV  component  contains  14  TOs,  such  as  M243  Smoke  Grenade  Launcher,  M60  Machine 
Gun,  and  others.  Each  TO  has  between  1  and  4  specific  performance  measures.  The 
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HMMWV  component  contains  12  Training  Objectives,  such  as  Stow  BII  on  the  HMMWV, 
Conduct  PMCS  on  a  HMMWV,  and  others.  Each  has  between  1  and  4  performance 
measures.  These  grouped  scores  for  HMMWV  and  for  ITV  still  exhibited  extremely 
low  variability,  extreme  negative  skew,  and  large  kurtosis. 

We  then  made  an  effort  to  retain  as  many  cases  as  possible  when  computing 
the  TO  and  HMMWV  and  ITV  summary  measures  by  prorating.  For  example,  for  TO  1, 
which  had  16  performance  measures,  we  defined  the  TO  1  summary  measure  as  the  sum 
of  the  16  scores  divided  by  16.  If  a  case  had  scores  on  15  of  the  16  measures, 
we  recoded  the  missing  data  value  as  0,  computed  the  sum,  and  divided  by  15  (one 
less  than  the  16  measures,  due  to  the  missing  score).  The  sample  contained  714 
HMMWV  soldiers  and  385  ITV  soldiers. 

Results  and  Discussion.  We  present  results  separately  for  HMMWV  and  for 
ITV  students. 

HMMWV:  Table  98  presents  descriptive  statistics  on  the  IIH  variables  for 
the  HMMWV  students. 

We  conducted  factor  analysis  on  TOs  1,  2,  4,  5,  and  7;  the  HUMMWV 
sum;  the  10  shots  each  for  events  1-3;  and  the  10  shots  for  the  TOW  event  1.  The 
correlation  matrix  of  these  variables  had  a  determinant  of  zero,  which  may 
indicate  collinearity  among  two  or  more  variables.  However,  examination  of  the 
correlation  matrix  did  not  reveal  any  variables  having  linear  combinations  of 
other  variables.  TO  4  had  a  communal ity  of  .99,  but  if  TO  4  were  perfectly 
predictable  from  other  variables,  it  would  have  a  communality  of  exactly  1.00. 
Further,  even  with  the  TO  variables  and  the  HMMWV  sum  removed,  the  correlation 
matrix  of  the  event  shots  still  had  a  determinant  of  zero.  None  of  these  other 
communal ities  even  approached  1.00.  Thus,  we  present  the  following  conclusions 
as  tentative  under  the  above  proviso  of  the  singular  correlation  matrix. 
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TjMc 


Descnntive  Sotistics  tor  IIH  Vanahles 


V’anable  Mean  S:d  Dev  Skc^-ncss  Kunosis  Minimum  Maximum 


TO  1 

2.979 

.148 

TO  2 

2.971 

.143 

TO  4 

2.974 

.108 

TO  5 

2.981 

.151 

TO  7 

2.958 

.088 

Ev  1  Sh  1 

47.216 

20.780 

Ev  1  Sh  2 

49^51 

20.852 

Ev  1  Sh  3 

50.097 

21.268 

Ev  1  Sh  4 

49J33 

20.451 

Ev  1  Sh  5 

49.733 

20.638 

Ev  1  Sh  6 

50J13 

20303 

Ev  1  Sh  7 

50.721 

21.W5 

Ev  1  Sh  8 

50.556 

20.275 

Ev  1  Sh  9 

50.677 

20.993 

Ev  1  Sh  10 

50.141 

21.476 

E.  ■  Sh  1 

56.154 

r.984 

Ev  2  Sh  2 

57.463 

19.331 

Ev  2  Sh  3 

57374 

19.150 

Ev  2  Sh  4 

57.812 

IS.  140 

Ev  2  Sh  5 

58.062 

17.988 

Ev  2  Sh  6 

57.819 

18.685 

Ev  2  Sh  7 

57328 

18,954 

Ev  2  Sh  8 

57.867 

18.781 

Ev  2  Sh  9 

58.062 

18.472 

Ev  2  Sh  10 

55.751 

19300 

Ev  3  Sh  1 

64369 

13.305 

Ev  3  Sh  2 

65309 

12.396 

Ev  3  Sh  3 

65.860 

12.636 

Ev  3  Sh  4 

66.034 

12.659 

Ev  3  Sh  5 

65393 

13.046 

Ev  3  Sh  6 

65322 

12.619 

Ev  3  Sh  7 

65.619 

12.393 

Ev  3  Sh  8 

65.692 

13304 

Ev  3  Sh  9 

66.198 

12.427 

Ev  3  Sh  10 

65.835 

13.155 

HMMWV  Sum 

2.997 

.020 

TOW  Ev  1  Sh  1 

65.252 

12.108 

TOW  Ev  1  Sh  2 

65349 

12345 

TOW  Ev  1  Sh  3 

65.907 

12.010 

TOW  Ev  1  Sh  4 

66.750 

12.707 

TOW  Ev  1  Sh  5 

67.174 

12.011 

TOW  Ev  1  Sh  6 

66301 

12.211 

TOW  Ev  1  Sh  7 

66378 

13.065 

TOW  Ev  1  Sh  8 

66.914 

12.712 

TOW  Ev  1  Sh  9 

67.253 

12.940 

TOW  Ev  1  Sh  10 

67.342 

12321 

Note,  n  =  714. 


•11.370 

140.740 

1.000 

3.000 

-8.876 

106.179 

1.000 

3.000 

-10.374 

123.676 

1333 

3.000 

-10.961 

132.383 

1.000 

3.000 

-6.124 

58.905 

2.000 

3.000 

-.392 

-.210 

.000 

100.000 

-.501 

-.078 

.000 

100.000 

-.618 

-.119 

.000 

91.000 

•324 

-.132 

.000 

100.000 

-.561 

-.011 

.000 

98.000 

-.705 

.203 

.000 

96.000 

-  722 

.063 

.000 

94.000 

-.040 

.070 

.000 

98.000 

-598 

-.088 

.000 

100.000 

-620 

- 177 

.000 

99.000 

-  *00 

.635 

.000 

100,000 

-.898 

.948 

.000 

98.000 

-.940 

.851 

.000 

100.000 

-.930 

1.054 

.000 

95.000 

-.937 

1.072 

.000 

100.000 

-.861 

.984 

.000 

100.000 

-1.021 

1.180 

.000 

98.000 

-1.078 

1.470 

.000 

96.000 

-.905 

1.183 

.000 

100.000 

-.9(M 

.807 

.000 

97.000 

-1.059 

4.075 

.000 

100.000 

-1.115 

4.792 

.000 

96.000 

-1.102 

4328 

.000 

100.000 

-.979 

3.920 

.000 

100.000 

-1.247 

4.939 

.000 

95.000 

-.671 

1.992 

.000 

99.000 

-323 

1.218 

10.000 

98.000 

-1.119 

3.754 

.000 

98.000 

-350 

1.608 

4.000 

98.000 

-.910 

3,318 

.000 

100.000 

-6.782 

47.380 

2.833 

3.000 

-1.148 

3.644 

.000 

98.000 

-1.162 

4.270 

.000 

95.000 

-1.301 

4.892 

.000 

100.000 

-1.452 

5361 

.000 

98.000 

-1.239 

3.977 

.000 

92.000 

-1.176 

3.989 

.000 

98.000 

-1360 

5.266 

.000 

94.000 

-1.241 

4.106 

.000 

97.000 

-1.217 

4.013 

.000 

99.000 

-1.008 

3.478 

.000 

100.000 

154 


Factor  analysis  indicates  five  factors.  Table  99  shows  the  communal ities 
and  factor  loadings,  while  Table  100  shows  the  factor  score  coefficients.  Factor 
1  clearly  consists  of  the  10  shots  for  Event  3.  The  10  shots  for  Event  1  defined 
Factor  2,  while  the  10  shots  for  Event  2  defined  Factor  3.  Factor  4. consists  of 
the  10  shots  for  the  first  TOW  Event.  Finally,  the  fifth  factor  consists  of  TO 
1,  TO  4,  TO  5,  and  TO  7. 

In  order  to  represent  important  dimensions  of  the  HMMWV  student  performance 
in  the  IIH  school,  the  researchers  developed  composite  variables  based  on  the 
empirical  factors  which  emerged  from  the  factor  analysis.  Total  score  on  the  10 
shots  in  Event  3  constituted  the  most  important  dimension.  This  total  score 
usually  corresponds  to  the  qualification  score,  because  by  then  the  students  have 
practiced  shooting  enough  that  their  scores  have  increased  sufficiently.  The 
shots  for  Events  1  and  2  and  the  TOW  Event  1  loaded  on  Factors  2,  3,  and  4, 
respectively.  Finally,  the  measure  of  training  objective  performance  consisted 
of  the  average  (on  a  0-3  scale)  of  the  prorated  training  objective  scores  for 
which  the  soldiers  exhibited  variability  (TOs  1,  2,  4,  5,  and  7). 
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Table 

rommunaimcs  and  Factor  Lx)adings  for  IIH  HMMV»*V  Variables 


\’anabic 

Communaiity 

1 

- 

Factor  Loadings 

3  4 

5 

TO  1 

.990 

.032 

.031 

.019 

-.003 

.994 

TO  2 

.035 

.095 

-.013 

.130 

-.016 

.091 

TO  4 

.989 

.035 

.038 

.028 

.001 

.993 

TO  5 

.949 

.019 

.cm 

.046 

-.005 

.972 

TO  7 

.457 

.069 

.097 

.039 

-.021 

.664 

Ev  1  Sh  1 

.363 

.128 

.567 

.152 

.037 

.027 

Ev  1  Sh  2 

.429 

.128 

.613 

.189 

.035 

.008 

Ev  1  Sh  3 

.497 

.209 

.626 

.230 

.075 

.058 

Ev  1  Sh  4 

.546 

.141 

.679 

.250 

.040 

.020 

Ev  1  Sh  5 

.552 

.157 

.700 

.190 

.033 

-.006 

Ev  1  Sh  6 

.528 

.083 

.689 

.208 

.054 

.025 

Ev  1  Sh  7 

.523 

.126 

.686 

.184 

.051 

.016 

Ev  1  Sh  8 

.494 

.177 

.651 

.194 

.009 

.040 

Ev  1  Sh  9 

.494 

.238 

.621 

.209 

.054 

.068 

Ev  1  Sh  10 

.471 

.225 

.618 

.194 

-.002 

.041 

Ev  2  Sh  1 

419 

.180 

.326 

.518 

.063 

.092 

Ev  2  Sh  2 

.484 

.227 

.308 

.580 

.019 

.028 

Ev  2  Sh  3 

S03 

.164 

.294 

.620 

.034 

.066 

Ev  2  Sh  4 

SiS 

.244 

.295 

.628 

.063 

.018 

Ev  2  Sh  5 

S50 

.221 

.338 

.620 

.032 

.047 

Ev  2  Sh  6 

.575 

.248 

.262 

.661 

.077 

-.047 

Ev  2  Sh  7 

.605 

.235 

.284 

.685 

.000 

.012 

Ev  2  Sh  8 

.494 

.309 

.230 

.588 

.011 

.010 

Ev  2  Sh  9 

J26 

.308 

.233 

.612 

.033 

-.042 

Ev  2  Sh  10 

.516 

.326 

.274 

.578 

-.013 

-.003 

Ev  3  Sh  1 

.475 

.623 

.168 

.239 

.037 

-.007 

Ev  3  Sh  2 

.452 

.619 

.157 

.207 

.025 

.019 

Ev  3  Sh  3 

i26 

.670 

.184 

.204 

-.032 

-.008 

Ev  3  Sh  4 

iOO 

.658 

.129 

.209 

.083 

.013 

Ev  3  Sh  5 

.560 

.694 

.196 

.193 

.057 

-.018 

Ev  3  Sh  6 

.445 

.618 

.183 

.156 

.031 

.065 

Ev  3  Sh  7 

511 

.686 

.148 

.185 

.027 

.003 

Ev  3  Sh  8 

523 

.681 

.126 

.159 

.109 

.077 

Ev  3  Sh  9 

530 

.693 

.145 

.161 

.021 

.053 

Ev  3  Sh  10 

.501 

.667 

.149 

.173 

.026 

.054 

HMMWV  Sum 

.009 

-.061 

-.026 

-.037 

.058 

-.007 

TOW  Evl  Shi 

.281 

.012 

.086 

.001 

^23 

-.004 

TOW  Evl  Sh2 

.372 

.021 

.047 

.012 

.606 

-.037 

TOW  Evl  Sh3 

.462 

.060 

.003 

.061 

.674 

.013 

TOW  Evl  Sh4 

.432 

.045 

.039 

.049 

.653 

.003 

TOW  Evl  Sh5 

539 

-.019 

.001 

.040 

.733 

.020 

TOW  Evl  Sh6 

All 

-.027 

.038 

.019 

.651 

.032 

TOW  Evl  Sh7 

Ale 

.097 

.003 

.038 

.682 

.017 

TOW  Evl  Sh8 

.490 

.049 

.048 

-.017 

.693 

-.065 

TOW  Evl  Sh9 

.436 

.045 

.032 

.001 

.658 

.010 

TOW  Evl  SIO 

.507 

.004 

.025 

.025 

.711 

-.020 
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Tabic  K'O 

Factor  ^core  Coefficients  for  11 H  HNtMV»V  Variables 


Factor  Score  Coefficients 


N'anablc 

1 

- 

4 

5 

TO  1 

.024 

-.016 

-.113 

-.037 

“795 

TO  2 

.006 

-.012 

.013 

-.002 

.008 

TO  4 

.002 

-.024 

.047 

.053 

.402 

TO  5 

..054 

-.003 

.069 

-.006 

.112 

TO  7 

.006 

.014 

-.005 

-.007 

-.005 

Ev  1  Sh  1 

-.012 

.105 

-.038 

-.001 

.016 

Ev  1  Sh  2 

-.022 

.130 

-.038 

.001 

-.006 

Ev  1  Sh  3 

-.005 

.135 

-.039 

.006 

-.003 

Ev  1  Sh  4 

-.036 

.167 

■ 

o 

-.003 

-.004 

Ev  1  Sh  5 

-.020 

.183 

-.067 

-.008 

-.001 

Ev  1  Sh  6 

-.042 

.166 

-.W3 

-.002 

-.003 

Ev  1  Sh  7 

-.031 

.172 

-.057 

.001 

-.010 

Ev  1  Sh  8 

..010 

.151 

-.054 

-.010 

-.025 

Ev  1  Sh  9 

.004 

.130 

-.019 

.005 

-.005 

Ev  1  Sh  10 

.000 

.132 

-.049 

-.014 

.009 

Ev  2  Sh  1 

-.030 

.007 

.097 

.002 

-.005 

Ev  :  Sh  2 

-.026 

-.017 

.136 

-.007 

.027 

E-.  2  Sh  3 

-.052 

-.020 

.168 

-.004 

-.001 

Ev  2  Sh  4 

-.032 

-.032 

.174 

.004 

-.014 

Ev  2  Sh  5 

..038 

-.011 

.158 

-.004 

-.009 

Ev  2  Sh  6 

-.033 

-.052 

.198 

.013 

-.007 

Ev  2  Sh  7 

.,W8 

-.Wl 

.219 

-.011 

-.001 

Ev  2  Sh  8 

-.003 

-.039 

.130 

-.007 

-.000 

Ev  2  Sh  9 

-.009 

-.(M7 

.163 

-.000 

-.008 

Ev  2  Sh  10 

.000 

-.029 

.132 

-.017 

-.004 

Ev  3  Sh  1 

.123 

-.021 

-.017 

-.005 

-.012 

Ev  3  Sh  2 

.116 

-.017 

-.022 

-.004 

.009 

Ev  3  Sh  3 

.149 

-.015 

-.036 

-.022 

-.004 

Ev  3  Sh  4 

.136 

-.032 

-.025 

.011 

-.006 

Ev  3  Sh  5 

.168 

-.012 

-.053 

.002 

.001 

Ev  3  Sh  6 

.116 

-.006 

-.039 

-.003 

,004 

Ev  3  Sh  7 

.159 

-.027 

-.042 

-.005 

.003 

Ev  3  Sh  8 

.154 

-.025 

-.042 

.013 

-.020 

Ev  3  Sh  9 

.165 

-.020 

-.053 

-.008 

.005 

Ev  3  Sh  10 

.141 

-.018 

-.041 

-.007 

-.006 

HMMWV  Sum 

-.005 

-.002 

.001 

.006 

-.011 

TOW  Evl  Shi 

-.005 

.014 

-.017 

.080 

-.012 

TOW  E  1  Sh2 

-.004 

.002 

-.005 

.111 

.002 

TOW  Evl  Sh3 

.002 

-.016 

.005 

.134 

-.011 

TOW  Evl  Sh4 

-.006 

-.007 

.004 

.126 

.010 

TOW  Evl  Sh5 

-.021 

-.020 

.019 

.178 

.018 

TOW  Evl  Sh6 

-.015 

.003 

-.002 

.122 

.015 

TOW  Evl  Sh7 

.009 

-.018 

.001 

.139 

-.014 

TOW  Evl  Sh8 

.003 

.007 

-.024 

.150 

.003 

TOW  Evl  Sh9 

.005 

-.002 

-.015 

.122 

.002 

TOW  Evl  SIO 

-.013 

-.008 

.004 

.161 

-.004 

Table  101  provides  descriptive  statistics  on  these  composites,  and  Table 
102  displays  their  correlations  with  the  factors  and  among  the  composites.  Table 
101  presents  composite  reliabilities  calculated  by  two  distinct  methods.  The 
first  column  presents  reliabilities  calculated  using  communality  estimates  from 
factor  analysis.  The  second  method  presents  reliabilities  for  the  Events  under 
the  assumption  that  each  shot  within  an  Event  constitutes  an  independent 
replication.  The  researchers  computed  the  mean  of  the  inter-shot  (interitem) 
correlations  within  each  Event.  The  reliabilities  computed  from  the  communality 
estimates  (.83  to  .92)  exceeded  the  average  inter-shot  correlations  (.43  to  .51). 

In  Table  102.  the  strong  correlation  (from  .90  to  1.00)  between  each  factor 
and  its  corresponding  composite  supports  use  of  the  composite  variables. 
Further,  the  relatively  strong  correlations  between  the  totals  for  Events  1  to 
3  (.63,  .46,  .58)  indicate  that  the  three  Events  measure  some  common  higher-order 
factor.  We  factor  analyzed  the  correlation  matrix  of  the  three  Event  totals  and 
found  one  strong  factor.  Event  1,  2,  and  3  totals  had  the  following  factor 
properties:  communal ities:  .501,  .791,  and  .424;  factor  loadings:  .708,  .889, 
and  .651;  and  factor  score  coefficients:  .217,  .652,  and  .173. 


Table  101 

Descriptive  Statistics  for  IIH  HMMWV  Comtxjsite  Variables 


Composite 

Rel* 

RcP 

Mean 

SD 

Skew 

Kunosis 

Minimum 

Maximum 

TO® 

,90 

N/A 

2.974 

.108 

-10374 

123.676 

1533 

3.000 

Ev  1  Total 

.90 

.48 

498.844 

152.727 

-.496 

.109 

0.000 

844.000 

Ev  2  Total 

.92 

JSl 

574.098 

140.664 

-.718 

.756 

25.000 

910.000 

Ev  3  Total 

.91 

.50 

656.432 

95.332 

-.986 

4511 

70.000 

923.000 

TOW  Ev  1  Total 

.89 

.43 

665557 

89536 

-1500 

5.883 

6ZOOO 

874.000 

^These  are  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^Mean  inter-shot  correlation  within  each  Event. 

®TO:  (TO  1  +  TO  2  +  TO  4  +  TO  5  +  TO  7)/5 
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Tabic  lO: 

Correlations  Among  llH  Composites  and  Factors 


Composite 

Fac  1 

Kac2 

Fac  3 

Fac  4 

Fac  5 

TO 

Evl 

Ev2 

Ev3 

Factor  2 

.03 

Factor  3 

.07 

.09 

Factor  4 

.01 

.01 

,00 

Factor  5 

.00 

.00 

-.00 

-.00 

TO* 

.06 

.05 

.07 

-.01 

.95 

Ev  1  Total 

.24 

.95 

.30 

.06 

.04 

.11 

Ev  2  Total 

.35 

.41 

.90 

.04 

.02 

.11 

.63 

Ev  3  Total 

.96 

.23 

.28 

.06 

.03 

.11 

.46 

.58 

TOW  Ev  1  Total  .04 

.05 

.04 

1.00 

-.00 

-.01 

.10 

.10 

.10 

•TO:  (TO  1 

+  TO  2  +  TO  4 

+  T0  5 

+  TO  7)/5 

ITV:  Table  103  shows  the  descriptive  statistics  on  the  ITV  variables 
included  in  the  factor  analysis.  The  researchers  eliminated  the  measure  for  TO 
6  due  to  lack  of  variability.  As  with  the  HMMWV  variables,  all  the  ITV  variables 
demonstrated  negative  skew,  indicating  a  ceiling  effect. 

Factor  analysis  indicates  7  factors,  with  the  communalities  and  loadings 
shown  in  Table  104  and  the  factor  score  coefficients  in  Table  105.  As  with  the 
HMMWV  data,  the  ITV  correlation  matrix  had  a  determinant  of  zero.  This  singular 
correlation  matrix  tempers  the  conclusions  derived  from  this  analysis.  The  shots 
for  Event  1  defined  the  primary  factor,  while  the  shots  for  Event  3  defined  the 
second  factor.  The  shots  for  the  first  ITV  Event  defined  the  third  factor,  while 
the  shots  for  Event  2  defined  the  fourth  factor.  Factor  5  consisted  of  the 
training  objectives,  while  factors  6  and  7  consisted  of  the  shots  for  ITV  Events 
3  and  2,  respectively. 
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Tabic  iL'3 

Descnrtivc  .Siatisiics  for  IIH  rP.*  Variables 


1 

\ 


I 


} 


\  anabic  Mean  Sid  Dev  Skeu-ness  Kurtosis  Minimum  Maximum 


TO  1 

2,996 

.024 

-8.494 

85.695 

2.688 

3,000 

TO  2 

2.984 

.096 

-7.391 

63.227 

2.000 

3.000 

TO  3 

2.996 

.028 

-7.607 

62.887 

2.714 

3.000 

TO  4 

2.997 

.021 

-7.540 

66.022 

2.750 

3.000 

TO  5 

2.994 

.042 

-7.914 

69.458 

2300 

3.000 

TO  7 

2.971 

.057 

-2,455 

7.309 

2.600 

3.000 

Ev  1  Sh  1 

46.070 

21.695 

-.453 

-.444 

.000 

100.000 

Ev  i  Sh  2 

48.419 

21.438 

-338 

-.303 

.000 

100.000 

Ev  1  Sh  3 

48.901 

21.821 

-366 

-.328 

.000 

90.000 

Ev  1  Sh  4 

49380 

21.784 

-.594 

-.120 

.000 

100.000 

Ev  1  Sh  5 

51.115 

19376 

-.671 

.089 

.000 

95.000 

Ev 1  Sh  6 

50.627 

20.475 

-.637 

.103 

.000 

100.000 

Ev  1  Sh  7 

49.817 

21345 

-.467 

-.368 

.000 

100.000 

Ev  1  Sh  8 

49.368 

21.138 

-.591 

-.161 

.000 

94.000 

Ev  1  Sh  9 

49.632 

21.109 

-362 

-.296 

.000 

92.000 

Ev  1  Sh  10 

49.890 

21.202 

-.735 

-.047 

.000 

100.000 

Ev  2  Sh  1 

55.676 

19.167 

-.983 

.929 

.000 

92.000 

Ev  :  Sh  2 

56.308 

1S.388 

-.987 

1.001 

.000 

95.000 

E'-  .  Sh  3 

59.640 

r.802 

-.937 

1.118 

.000 

98.000 

Ev  2  Sh  4 

57.862 

17,733 

-.916 

1.119 

.000 

91.000 

Ev  2  Sh  5 

58.728 

17.825 

-.978 

1.325 

.000 

95.000 

Ev  2  Sh  6 

58.065 

IS.  145 

-.837 

.722 

.000 

95.000 

Ev  2  Sh  7 

58.063 

1S.277 

-.895 

1.193 

.000 

97.000 

Ev  2  Sh  8 

57.783 

1S.882 

-.961 

1.046 

.000 

99.000 

Ev  2  Sh  9 

58.325 

18.604 

-.919 

1.044 

,000 

99.000 

Ev  2  Sh  10 

57.157 

19.487 

-.815 

.724 

.000 

97.000 

Ev  3  Sh  1 

65.806 

13338 

-.957 

2.691 

.000 

100.000 

Ev  3  Sh  2 

66.228 

13.158 

-.657 

1.797 

.000 

98.000 

Ev  3  Sh  3 

65.961 

13.112 

-.880 

2389 

.000 

96.000 

Ev  3  Sh  4 

65.364 

13.211 

-.952 

2.803 

.000 

95.000 

Ev  3  Sh  5 

65.678 

13.125 

-.867 

3.078 

.000 

93.000 

Ev  3  Sh  6 

67.197 

12.961 

-.704 

2.022 

5.000 

96.000 

Ev  3  Sh  7 

65.824 

13322 

-.734 

1.793 

8.000 

99.000 

Ev  3  Sh  8 

67.060 

14.052 

-1.223 

3.862 

.000 

95.000 

Ev  3  Sh  9 

66.433 

12.820 

-1.281 

4.835 

.000 

97,000 

Ev  3  Sh  10 

67.016 

14.012 

-1.018 

2.640 

5.000 

95.000 

nv  Sum 

2.999 

.007 

-6.331 

44.685 

2.929 

3.000 

nv  Ev  1  Sh  1 

50342 

23.117 

-.715 

-.193 

.000 

98,000 

nv  Ev  1  Sh  2 

53.008 

22310 

-388 

-.044 

.000 

98.000 

nv  Ev  1  Sh  3 

53.070 

22390 

-.717 

.079 

,000 

96.000 

nv  Ev  1  Sh  4 

52.854 

23.031 

-.635 

-.102 

.000 

100.000 

nv  Ev  1  Sh  5 

52.809 

22.083 

-.728 

.155 

.000 

100.000 

nv  Ev  1  Sh  6 

54,655 

22.028 

-.709 

.087 

.000 

99.000 

nv  Ev  1  Sh  7 

53.880 

22321 

-.674 

-.087 

,000 

100.000 

nv  Ev  1  Sh  8 

54335 

22.488 

-.704 

.116 

.000 

100.000 

nv  Ev  1  Sh  9 

53,256 

21.957 

-.736 

.229 

.000 

100.000 

nv  Ev  1  Sh  10 

52.726 

22.473 

-.707 

.051 

.000 

100.000 
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Table  103  (Continued) 


Variable 

Mean 

Sid  Dev 

SkewTicss 

Kunosis 

Minimum  Maximum 

rrv  Ev  2  Sh  1 

57.688 

1S.687 

-.846 

.697 

.000 

96.000 

nv  Ev  2  Sh  2 

58.122 

20.120 

..S67 

.867 

.000 

100.000 

nv  Ev  2  Sh  3 

59.482 

19.652 

-1.007 

.819 

.000 

92.000 

nv  Ev  2  Sh  4 

58.172 

20.909 

-i.cm 

.993 

.000 

96.000 

nVEv2Sh5 

58.201 

19.328 

-.808 

.639 

.000 

95.000 

rrv  Ev  2  Sh  6 

58347 

20.447 

-1.016 

1.145 

,000 

99.000 

nv  Ev  2  Sh  7 

59342 

18.845 

-.935 

1.018 

.000 

95.000 

rrv  Ev  2  Sh  8 

58.823 

20.213 

-.921 

.754 

.000 

98.000 

nv  Ev  2  Sh  9 

58.089 

20354 

-.887 

.801 

.000 

96.000 

rrv  Ev  2  Sh  10 

57.898 

19.251 

-.822 

.882 

.000 

97.000 

rrv  Ev  3  Sh  1 

68.219 

12,213 

-.521 

2.169 

.000 

100.000 

rrv  Ev  3  Sh  2 

69.204 

12.780 

-.931 

3.118 

.000 

100.000 

rrv  Ev  3  Sh  3 

68.339 

12.276 

-.997 

4.489 

.000 

98.000 

rrv  Ev  3  Sh  4 

68.232 

13.302 

-.697 

2.779 

.000 

99.000 

nv  Ev  3  Sh  5 

68.862 

12.262 

-.253 

-.003 

26.000 

95.000 

rrv  Ev  3  Sh  6 

67.465 

12.481 

-.718 

2.224 

5.000 

97.000 

rrV'  Ev  3  Sh  7 

68.269 

12.833 

-.572 

2.446 

.000 

100.000 

nv  Ev  3  Sh  8 

68.256 

12.832 

-.668 

1.962 

.000 

96.000 

nv  Ev  3  Sh  9 

69.407 

12.998 

-.472 

.743 

20.000 

100.000 

nv  Ev  3  Sh  10 

67.911 

13.599 

-.962 

2.903 

.000 

96.000 

Note,  n  =  385. 

Table  104 

Communalities  and  Factor  Loadines  for  IIH  ITV' Variables 

Variable 

Communality 

1 

Factor  Loadings 

2  3  4  5 

6 

7 

TOl 

.792 

.061 

.056 

.026 

-.002 

.886 

.011 

.017 

TO  2 

.050 

-.075 

-.019 

-.055 

.071 

.181 

,056 

.003 

TO  3 

.936 

.053 

.065 

.011 

-.034 

.963 

.004 

.015 

TO  4 

.951 

.056 

.024 

.041 

-.021 

.971 

.016 

.032 

TO  5 

.776 

.022 

.094 

-.004 

-.063 

.873 

.004 

.008 

TO  7 

.083 

-.005 

-.030 

■ 

o 

.047 

.271 

-.006 

-.080 

Ev  1  Sh  1 

316 

.673 

.159 

-.030 

.175 

.069 

.006 

.039 

Ev  1  Sh  2 

325 

.705 

.095 

o 

p 

.102 

.032 

.079 

.024 

Ev  1  Sh  3 

.619 

.759 

.137 

.034 

.129 

.007 

.082 

.001 

Ev  1  Sh  4 

365 

.705 

.121 

.047 

.196 

.011 

,074 

.082 

Ev  1  Sh  5 

.491 

.669 

,068 

.001 

.167 

-.093 

.051 

-.014 

Ev  1  Sh  6 

370 

.707 

.093 

.061 

.237 

.014 

.019 

-.024 

Ev  1  Sh  7 

349 

.693 

.089 

.120 

.200 

-.056 

.040 

.052 

Ev  1  Sh  8 

383 

.658 

.124 

.089 

.334 

-.008 

.073 

.098 

Ev  1  Sh  9 

346 

.685 

.117 

.041 

.231 

.069 

.038 

.046 

Ev  1  Sh  10 

.484 

.647 

.076 

.034 

.227 

-.067 

.007 

.050 
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IjMc  lf.4  (Continued) 


\*ariable  Communaliiy 

1 

Factor  Loadings 

2  3  4  5 

6 

1 

Ev  2  Sh  1 

.392 

.425 

.113 

.012 

.435  .089 

.027 

.027 

Ev  2  Sh  2 

i06 

.426 

.107 

.040 

349  .061 

.052 

-.064 

Ev  2  Sh  3 

J36 

.338 

.130 

.024 

.633  .057 

.020 

.022 

Ev  2  Sh  4 

.550 

.360 

.202 

.038 

.609  -.035 

.080 

-.014 

Ev  2  Sh  5 

-561 

.351 

.157 

.014 

,626  .059 

.120 

,052 

Ev  2  Sh  6 

303 

.429 

.209 

.056 

315  .029 

.082 

.012 

Ev  2  Sh  7 

.602 

.290 

.231 

.025 

.677  .002 

.064 

.037 

Ev  2  Sh  8 

.612 

.294 

.290 

-.014 

.664  .016 

.012 

.001 

Ev  2  Sh  9 

.609 

.344 

.199 

.029 

.668  .001 

.051 

.035 

Ev  2  Sh  10 

358 

.319 

.215 

.101 

.625  -.010 

-.007 

.091 

Ev  3  Sh  1 

.358 

.287 

.490 

.024 

.188  -.016 

-.001 

.012 

Ev  3  Sh  2 

356 

.225 

.679 

.136 

.151  -.029 

.048 

.007 

Ev  3  Sh  3 

355 

.120 

.712 

.130 

.109  -.025 

-.017 

-.059 

Ev  3  Sh  4 

303 

.144 

.650 

.176 

.137  -.097 

-.018 

-.010 

Ev  3  Sh  5 

.5W 

.173 

.664 

.037 

.158  .012 

-.010 

.080 

E\'  3  Sh  6 

.579 

.W9 

.744 

.041 

.128  -.017 

.W8 

.042 

Ev  3  Sh  7 

.575 

.C48 

.742 

-.011 

.083  .081 

.005 

.094 

Ev  3  Sh  8 

.591 

.116 

.746 

.034 

.106  .071 

.052 

.037 

Ev  3  Sh  9 

.472 

.074 

.658 

.035 

.113  .065 

.086 

.088 

Ev  3  Sh  10 

302 

.087 

.681 

-.003 

.135  .022 

.039 

.100 

rrv'  Sum 

.017 

.040 

-.074 

.022 

.052  -.032 

-.016 

-.074 

rrv  Ev  1  Sh  1 

.314 

.055 

.005 

.512 

.000  .037 

.110 

.188 

rrv  Ev  1  Sh  2 

.319 

-.039 

.048 

-518 

-.007  -.054 

.069 

.197 

nv  Ev  1  Sh  3 

.373 

.067 

.068 

.580 

.018  -.022 

.061 

.149 

rrv  Ev  j  Sh  4 

.430 

.115 

.081 

.608 

.081  -.073 

.119 

.119 

nv  Ev  1  Sh  5 

.425 

.010 

-.023 

.607 

.013  .032 

.149 

.182 

nv  Ev  1  Sh  6 

.445 

-.011 

.072 

398 

,082  -.016 

.098 

.257 

rrv  Ev  1  Sh  7 

.465 

.049 

.098 

.661 

.009  -.004 

.102 

.070 

rrv  Ev  1  Sh  8 

.400 

-.021 

.027 

390 

.028  .042 

.142 

.166 

rrv  Ev  1  Sh  9 

.360 

.084 

.038 

340 

.037  .054 

.203 

.120 

rrv  Ev  1  Sh  10 

.331 

-.004 

.051 

316 

-.039  -.066 

.138 

.194 

rrv  Ev  2  Sh  1 

.315 

.014 

.018 

.332 

.025  .014 

.095 

.441 

rrv  Ev  2  Sh  2 

.305 

-.013 

.031 

,323 

p 

r- 

8 

.181 

.406 

nV'  Ev  2  Sh  3 

.401 

.025 

.037 

.319 

.056  -.088 

.071 

530 

nv  Ev  2  Sh  4 

.392 

.090 

.029 

.291 

.052  -.038 

.153 

521 

nv  Ev  2  Sh  5 

.363 

.017 

-.044 

.201 

-.011  -.036 

.264 

500 

nv  Ev  2  Sh  6 

.451 

.068 

.083 

.241 

.053  .037 

.195 

583 

nv  Ev  2  Sh  7 

.453 

.065 

.076 

.253 

-.050  .005 

.172 

589 

nv  Ev  2  Sh  8 

.368 

.137 

.048 

.314 

.100  .050 

.194 

.445 

nv  Ev  2  Sh  9 

.487 

.006 

.085 

.307 

.029  -.038 

.242 

570 

nv  Ev  2  Sh  10 

.350 

.018 

.102 

.226 

.015  -.043 

.203 

.495 

nv  Ev  3  Sh  1 

.391 

-.001 

.031 

.141 

.053  -.055 

.574 

.188 

nv  Ev  3  Sh  2 

.261 

.053 

.001 

.123 

.026  .029 

.479 

.112 

nv  Ev  3  Sh  3 

.355 

.048 

.129 

.073 

.026  -.003 

^65 

.101 

nv  Ev  3  Sh  4 

.361 

.031 

.005 

.073 

.085  .034 

J86 

.056 

nv  Ev  3  Sh  5 

.312 

.050 

.072 

.104 

-.025  .050 

501 

.198 

nv  Ev  3  Sh  6 

320 

-.005 

.034 

.047 

.116  .012 

.546 

.067 

nv  Ev  3  Sh  7 

.332 

,077 

-.077 

.149 

.011  -.004 

.531 

.125 

nv  Ev  3  Sh  8 

.390 

.040 

.025 

.159 

.017  .043 

59S 

.078 

nv  Ev  3  Sh  9 

.329 

.049 

.023 

.169 

-.026  -.021 

532 

.119 

nv  Ev  3  Sh  10 

.343 

.037 

-.023 

.080 

-.012  .028 

571 

.089 

162 


Tabic  105 

Factor  Ncort  Coefficients  for  11 H  TT\'  Variables 


\'anablc 

1 

Factor  Score  Coefficients 

2  3  4 

5 

6 

7 

TO  1 

-.003 

•>42 

-.012 

-.003 

-.192 

.004 

-.028 

TO  2 

-.010 

.003 

-.014 

.008 

-.003 

.013 

.004 

TO  3 

.070 

.009 

.033 

-.082 

.626 

-.030 

-.014 

TO  4 

-.019 

-.091 

.043 

.038 

.737 

-.020 

.060 

TO  5 

-.037 

.(WO 

-.034 

.004 

-.190 

.016 

.004 

TO  7 

-.006 

-.003 

.006 

.011 

-.008 

.007 

-.021 

Ev  1  Sh  I 

.123 

-.000 

-.030 

-.055 

.008 

-.022 

.028 

Ev  1  Sh  2 

.146 

-.002 

-.021 

-.089 

.004 

.020 

-.001 

Ev  1  Sh  3 

.214 

.003 

-.002 

-.130 

.002 

.016 

-.032 

Ev  1  Sh  4 

.144 

-.014 

-.016 

-.062 

-.012 

.001 

.026 

Ev  1  Sh  5 

.110 

-.013 

-.011 

-.CWO 

-.(W8 

.007 

-.016 

Ev  1  Sh  6 

.149 

-.024 

.025 

-.(W8 

.025 

-.011 

-.040 

Ev  1  Sh  7 

.146 

-.017 

.021 

-.067 

-.037 

-.020 

-.005 

Ev  1  Sh  8 

.116 

-.015 

-.'302 

-.008 

-.029 

-.004 

.021 

Ev  1  Sh  9 

.123 

-009 

-.010 

-.036 

-.017 

-.006 

.009 

Ev  1  Sh  10 

.103 

-.022 

.'301 

-.032 

-.014 

-.019 

.012 

P*.  :  Sh  1 

.021 

-.016 

-.011 

.064 

.010 

-.006 

.010 

L .  2  Sh  2 

.009 

-.026 

.020 

.115 

.020 

.005 

-.(MS 

Ev  2  Sh  3 

-.028 

-.037 

-.002 

.169 

.008 

-.021 

.009 

Ev  2  Sh  4 

-.019 

-.009 

.004 

.145 

-.021 

.018 

-.035 

Ev  2  Sh  5 

-.031 

-.030 

-.024 

.169 

.007 

.029 

.013 

Ev  2  Sh  6 

.016 

-.008 

.005 

.079 

.009 

.011 

-.016 

Ev  2  Sh  7 

-.069 

-.009 

-.010 

.218 

-.016 

.002 

.003 

Ev  2  Sh  8 

-.068 

.005 

-.019 

.211 

.037 

-.008 

-.006 

Ev  2  Sh  9 

-.040 

-.021 

-.014 

.198 

.024 

-.003 

.005 

Ev  2  Sh  10 

-.035 

-.016 

.018 

.156 

-.000 

-.046 

.035 

Ev  3  Sh  1 

.014 

.050 

-.009 

-.006 

.014 

-.011 

.002 

Ev  3  Sh  2 

.013 

.141 

.038 

-.044 

.011 

.009 

-.050 

Ev  3  Sh  3 

-.016 

.153 

.046 

-.028 

-.022 

-.014 

-.073 

Ev  3  Sh  4 

-.004 

.124 

.CW9 

-.027 

-.037 

-.020 

-.046 

Ev  3  Sh  5 

-.002 

.115 

-.017 

-.020 

.024 

-.020 

.028 

Ev  3  Sh  6 

-.027 

.175 

-.025 

-.025 

-.053 

.015 

-.003 

Ev  3  Sh  7 

-.024 

.157 

-.044 

-.036 

.012 

-.013 

.040 

Ev  3  Sh  8 

-.013 

.186 

-.012 

-.053 

.012 

.022 

-.014 

Ev  3  Sh  9 

-.018 

.104 

-.022 

-.016 

-.001 

.012 

.016 

Ev  3  Sh  10 

-.025 

.128 

-.041 

-.016 

-.003 

.000 

.039 

nV  Sum 

.004 

-.005 

.013 

.003 

.004 

.002 

-.019 

nv  Ev  1  Sh  1 

.003 

-.011 

.108 

-.009 

-.007 

-.014 

-.013 

nV  Ev  1  Sh  2 

-.014 

-.009 

.106 

.008 

.012 

-.026 

-.010 

nv  Ev  1  Sh  3 

.003 

.002 

.143 

-.006 

-.020 

-.031 

-.042 

nv  Ev  1  Sh  4 

.006 

-.009 

.171 

-.001 

.007 

-.012 

-.077 

nv  Ev  1  Sh  5 

-.004 

-.021 

.152 

.007 

-.008 

-.007 

-.041 

nv  Ev  1  Sh  6 

-.026 

-.004 

.151 

.022 

.017 

-.032 

-.010 

nv  Ev  1  Sh  7 

-.002 

.002 

.211 

-.011 

.016 

-.015 

-.108 

nv  Ev  1  Sh  8 

-.014 

-.007 

.146 

.004 

-.015 

-.004 

-.052 

nv  Ev  1  Sh  9 

.002 

-.006 

.131 

-.004 

-.004 

.016 

-.061 

nv  Ev  1  Sh  10 

-.003 

-.008 

.106 

-.014 

.000 

-.004 

-.019 
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Table  i*j5  ( Continued  i 


\'anablc 

1 

Factor  Score  Cocfficicnis 

2  3  4 

5 

6 

7 

rrv  Ev  2  sh  1 

-.007 

-003 

.014 

.007 

-.007 

-.028 

.108 

rp.'  Ev  2  Sh  2 

-.009 

..'X4 

.023 

-.001 

-.008 

-.001 

.082 

m'  Ev  2  Sh  3 

-.009 

-.012 

.007 

.011 

-.023 

-.055 

.166 

nv  Ev  2  Sh  4 

.004 

-.012 

-.017 

.004 

.013 

-.024 

.165 

rr\'  Ev  2  Sh  5 

.005 

-.015 

-.035 

-.008 

.003 

.025 

.136 

rr\'  Ev  2  Sh  6 

.001 

.001 

-.045 

-.001 

1 

o 

o 

-.018 

.206 

nv  Ev  2  Sh  7 

.009 

.*002 

-.041 

-.024 

.004 

-.023 

.209 

rrv'  Ev  2  Sh  8 

.003 

-.012 

.004 

.010 

-.016 

-.000 

.099 

rrv'  Ev  2  Sh  9 

-.014 

.003 

-.022 

-.004 

-.018 

-.002 

.197 

rrv'  Ev  2  Sh  10 

-.005 

.007 

-.023 

-.003 

.0L5 

.000 

.133 

nv  Ev  3  Sh  1 

-.019 

.004 

-.018 

.002 

.014 

.160 

-.007 

rrv  Ev  3  Sh  2 

-.003 

-.010 

-.008 

.003 

-.000 

.106 

-.014 

ITV'  Ev  3  Sh  3 

-.004 

.019 

-.031 

-.021 

.011 

.159 

-.025 

rrv  Ev  3  Sh  4 

-.018 

-.009 

-.020 

.016 

.029 

.172 

-.050 

rrV'  Ev  3  Sh  5 

.001 

.001 

-.027 

-.014 

.002 

.119 

.007 

ITV'  Ev  3  Sh  6 

-.022 

.'002 

-.026 

.020 

-.019 

.151 

-.033 

rrv  Ev  3  Sh  7 

.004 

-.018 

-.007 

-.009 

-.008 

.132 

-.019 

ITV’  Ev  3  Sh  S 

-.004 

-•j02 

.002 

o 

o 

.014 

.174 

-.056 

rrv  Ev  3  Sh  9 

.003 

..X2 

-.004 

-.019 

-.014 

.134 

-.030 

ITV’  Ev  3  Sh  10 

-.002 

.004 

-.028 

-.011 

-.030 

.161 

-.036 

To  examine  whether  simpler  composite  variables  could  adequately  represent 
the  dimensions  of  school  performance  illustrated  by  the  empirical  factors,  the 
researchers  developed  summary  composite  measures.  Table  106  displays  descriptive 
statistics  on  these  composites,  while  Table  107  shows  correlations  between  the 
composites  and  the  empirical  factors  and  the  composite  intercorrelations.  The 
researchers  computed  reliability  estimates  for  the  Event  totals  in  two  different 
ways.  First,  we  computed  composite  reliability  using  the  communality  estimates 
from  the  factor  analysis.  Second,  we  computed  the  average  inter-shot  correlation 
within  each  of  the  six  Events,  under  the  assumption  that  each  shot  replicates  the 
others  within  the  Event.  The  reliabilities  based  on  commonalities  (.83  to  .92) 
consistently  exceeded  those  based  on  intercorrelations  (.33  to  .53). 
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Tabic 

Dcscnrtive  Statistics  for  rT\*  romposne  V'ariablcs 


Composite 

Reliab* 

Rcliab^ 

Mean 

Sid  Dev 

Skewness 

Runosis 

Minimum  Maximum 

TO'^ 

.88 

N/A 

2.990 

.030 

-5.426 

36.456 

2.719 

3.000 

Ev  1  Total 

.92 

.53 

494.045 

160.979 

-.427 

-.297 

.000 

873.000 

Ev  2  Total 

.92 

577.903 

139345 

-.822 

1302 

29.000 

935.000 

Ev  3  Total 

.91 

SO 

663.483 

98.640 

-386 

1.165 

308.000 

904.000 

ITV  Ev  1  Total 

.86 

.37 

531.136 

147.359 

-369 

.291 

.000 

897.000 

m'  Ev  2  Total 

.86 

.37 

584363 

130.302 

-.772 

1.106 

73.000 

854.000 

nv  Ev  3  Total 

.83 

.33 

684.164 

80.443 

-.893 

4.832 

205.000 

906.000 

^These  are  composites  based  on  communaiity  estimates  and  the  procedures  described  in  the  text. 

^Mean  inier-shot  correlation  within  each  Event. 

^TO:  (TO  1  +  TO  2  +  TO  3  +  TO  4  +  TO  5  +  TO  7)/6 

Table  107 

Correlations  Among  IIH  nr\'  Composites  and  Factors 

Composite 

1 

2 

3 

Factor 

4 

5 

6 

7 

TO 

Ev  1 

Ev2 

Ev3 

nv  Ev  1  nv  Ev  2 

Factor  2 

.01 

Factor  3 

.01 

.01 

Factor  4 

.09 

.04 

-.00 

Factor  5 

.01 

.00 

.00 

.01 

Factor  6 

.01 

-.00 

.04 

.00 

-.00 

Factor  7 

.00 

.01 

.13 

.09 

.00 

.08 

TO* 

-.01 

.03 

-.03 

.03 

.79 

.04 

-.02 

Ev  1 

.96 

.15 

.06 

.29 

-.00 

.07 

.05 

.00 

Ev2 

30 

.26 

.05 

.87 

.04 

.07 

.03 

.05 

.68 

Ev3 

.19 

.97 

.09 

.19 

.01 

.03 

.06 

.03 

.35 

.46 

ITVEvl 

.05 

.07 

.96 

.04 

-.01 

.20 

.29 

-.03 

.13 

.12 

.17 

nVEv2 

.07 

.07 

.47 

.05 

-.03 

.30 

.89 

-.03 

.15 

.14 

.17 

.62 

nVEv3 

.06 

.04 

.20 

.05 

.02 

.97 

.21 

.04 

.14 

.14 

.10 

.36  .46 

“TO;  (TO  1  +  TO  2  +  TO  3  +  TO  4  +  TO  5  +  TO  7)/6 


As  seen  in  Table  107,  the  strong  correlations  (i.e.,  .79  to  .97)  between 
each  composite  and  the  corresponding  factor  it  represents  support  the  use  of 
these  summary  criterion  composites.  The  totals  for  Events  1  to  3  demonstrate 
moderate  intercorrelations  (.35  to  .68),  as  do  the  totals  for  the  ITV  Events  1 
to  3  (.36  to  .62),  indicating  that  these  Event  totals  may  be  aggregated  to  form 
one  grand  total  for  Events  1  to  3  and  a  grand  total  for  ITV  Events  1  to  3. 
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To  quantitatively  assess  this  hypothesis,  we  factor  analyzed  the  Event 
totals  from  Table  107.  Table  108  displays  the  communal ities,  factor  loadings, 
and  factor  score  coefficients  from  this  higher-order  analysis.  The  factor 
loadings  clearly  indicate  two  factors  consisting  of  the  sum  (or  average)  of  the 
TOW  tracking  performance  Event  totals  and  the  sum  (or  average)  of  the  TOW 
simulator  tracking  Event  totals. 

We  also  computed  Pearson  correlation  coefficients  among  these  2  factors 
and  several  composites,  shown  in  Table  109.  The  average  of  the  3  Event  totals 
correlates  highest  with  Factor  1  (.93).  The  TOW  Qualification  score  correlates 
.42  with  Factor  1,  suggesting  that  the  TOW  Qualification  score  provides  a  measure 
of  maximal  performance  rather  than  typical  performance.  Similarly,  the  average 
ITV  event  total  correlates  highest  (.96)  with  Factor  2,  while  the  ITV 
Qualification  score  correlates  only  ,54  with  Factor  2, 


Table  108 

Communalities,  Factor  Loadings,  and  Factor  Score  Coefficients  for  IIH  ITV  Event  Totals 


Composite 


Factor  Loadings 

Communality  1  2 


Factor  Score  Coefficients 
1  2 


Ev  1  Total 

-517 

.712 

.096 

Ev  2  Total 

.900 

.947 

.054 

Ev  3  Total 

.246 

.474 

.146 

nv  Ev  1  Total 

.493 

.095 

.696 

nv  Ev  2  Total 

.785 

.094 

.881 

nv  Ev  3  Total 

.268 

.113 

.505 

.129  .008 

.844  -.102 

.050  .021 

.002  .232 

-.050  .695 

-.005  .113 


Table  109 

Correlations  Among  IlH  m'  Composites  and  Higher-Order  Faciors 


Composite^ 

TOW  Qual- 

row  Avg 

Faciorl 

ITV'  Qual  rrV  Avg  • 

TOW  Avg"^ 

56 

Factorl*^ 

.42 

.93 

nv  Qual® 

.10 

.11 

.07 

nv  Avg^ 

.16 

.21 

.13 

.64 

Factor2^ 

.15 

.12 

.02 

.54  .96 

^Thcse  are  composites  based  on  the  procedures  described  in  the  text. 
^'OW  Qual:  TOW  Qualification  Score. 

^TOW  Avg:  Av'crage  of  TOW  Events  1  through  3  Totals. 

^Factorl:  Factor  1  from  analysis  of  Event  Totals  shown  in  Table  108. 
®nv  Qual:  ITV  Qualification  Score. 

^rrv  Avg:  Average  of  TTV  Evrnts  1  through  3  Totals. 

*Factor2:  Factor  2  from  analysis  of  Event  Totals  shown  in  Table  108. 


Recommendations.  For  the  HMMWV  students,  use  the  average  of  the  totals 
for  Events  1  to  3  and  use  the  total  for  TOW  Event  1.  Also  use  the  average  of  the 
prorated  performance  in  the  training  objectives  in  the  first  part  of  training. 
For  ITV  students,  use  the  average  of  the  totals  for  Events  1  to  3,  and  the 
average  of  the  totals  for  ITV  Events  1  to  3.  Also  use  the  average  of  the 
prorated  performance  in  the  training  objectives  in  the  first  part  of  training. 
(The  Qualification  scores  provide  measures  of  maximal  performance  for  students 
in  both  programs.) 
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Tank  Crewman  (19K) 

Description  of  Variables.  RGI  received  student  data  that  included  an  Armor 
Stakes  score  sheet  and  score  sheets  for  three  training  modules  or  GATEs.  Due  to 
the  inconsistency  of  the  GATE  sheets  and  the  numerous  GATE  versions,  both  between 
and  within  classes,  RGI  only  entered  data  from  the  Armor  Stakes  scor€  sheet. 
(This  decision  had  the  support  of  Clint  Walker  at  the  Army  Research  Institute.) 

The  Armor  Stakes  is  an  end-of-course,  comprehensive  exam  that  includes 
tasks  from  GATEs  I,  II  and  III.  The  school  divides  the  Armor  Stakes  test  into 
five  separate  stations.  The  five  Armor  Stakes  versions  we  received  differed  in 
station  order  and  content.  In  Version  A,  the  school  administers  the 
Weapons/Communications  station  first  and  the  Nuclear,  Biological  and  Chemical 
Defence  (Nuc/Biol /Chem)  station  last.  In  Versions  B,  C,  D  and  E  the  school 
administers  the  Nuc/Biol /Chem  station  first  and  the  Weapons/Communications 
station  last.  The  school  administers  the  Maintenance  station.  Land 
Navigation/First  Aid  station,  and  Loader’s  Station  (stations  2,  3  and  4, 
respectively)  in  the  same  order  for  all  four  versions  of  Armor  Stakes.  The 
content  for  all  five  stations  differs  only  slightly  between  Versions  A,  B,  C,  D 
and  E.  The  list  below  provides  an  overview  of  version  similarities: 


Order  of 

Stations 

Version  A 

Versions  B.  C.  D  &  E 

1. 

Weapons/Communications 

1.  Nuc/Biol /Chem 

2. 

Maintenance 

2.  Maintenance 

3. 

Land  Navigation/First  Aid 

3.  Land  Navigation/First  Aid 

4. 

Loader’s  Station 

4.  Loader’s  Station 

5. 

Nuc/Biol /Chem 

5.  Weapons/Communications 
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For  all  five  Armor  Stakes  Versions,  a  station  contains  between  4  and  9 
tasks.  The  school  allows  a  soldier  2  attempts  to  pass  each  task,  assigning  a 
"GO"  or  "NO-GO"  accordingly.  The  soldiers  need  to  pass  75%  of  the  tasks 
administered  within  a  station,  otherwise  they  must  retake  that  station.  For  each 
Armor  Stakes  task,  RGI  assigned  a  1  for  passing  on  the  first  attempt,  a  2  for 
passing  on  the  second  attempt,  and  a  3  for  failing  a  task. 

Throughout  testing,  the  school  administered  Armor  Stakes  Versions  A  and  B 
to  different  classes  periodically.  The  school  administered  Armor  Stakes  Version 
C  to  a  group  of  soldiers  who  completed  GATE  III  on  04/07/91.  The  school  did  not 
indicate  when  these  soldiers  completed  the  Armor  Stakes.  In  addition,  the  school 
a'^mini stored  Armor  Stakes  Version  D  to  a  group  of  soldiers  on  05/28/91  and 
Version  E  to  three  soldiers  on  05/25-05/26/91.  The  school  indicated  that  their 
numerous  revisions  to  the  curricula  accounted  for  the  separate  versions.  All 
Armor  Stakes  tasks  are  performance  tasks.  The  school  records  no  Final  School 
Grade  (FSG)  or  measure  of  time  spent  in  training.  Student  action  codes  contained 
the  following  categories  of  disenrollment:  l*Medical  reason,  2=Failed  to 
qualify,  3*Discharge,  and  4=Unknown. 

The  school  administered  Armor  Stakes  Version  A  to  the  majority  of  soldiers. 
However,  on  some  occasions,  the  school  administered  tasks  6,  7,  and  8  from 
station  1  as  tasks  4,  5,  and  6,  in  station  4,  respectively,  and  task  7  from 
station  4  as  task  6  in  station  1.  The  form  variable  in  RGI*s  database  (i.e., 
l=most  prevalent  or  2=alternate  administration  order  as  described  above) 
indicates  which  order  the  soldiers  received  tasks.  The  school  indicated  that 
equipment  availability,  weather  conditions  and  instructors’  discretion  determine 
who  received  which  tasks  and  their  ordering.  Students  from  Armor  Stakes  Versions 
A  through  E  completed  the  tasks  shown  in  Tables  110  through  114,  respectively. 
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Table  110 


!9K  Armor  Stakes  Version  A 


Task  Station 


Statioci  l-Weapoos/Communicatioos 
Task  1:  Maintain  an  M9  pistol 
Task  2:  Load  an  M16A1  nflc 
Task  3:  Unload  an  M16A1  riHc 
Task  4:  Correct  malfunctions  of  an  M16A1  rifle 
Task  5:  Recognize  friendly  and  threat  vehicles  and  aircraft 
Task  6:  Prepare /operate  FM  radio  set 

Task  7:  Operate  inicrcommunication  set  AN/VIC-1  on  a  tracked  vehicle 
Task  8:  Send  a  radio  message 

Statioa  2-Maintcaance 

Task  1:  Extinguish  a  fire  on  an  Ml/MlAl  lank 

Task  2:  Vehicle  maintenance  1 171-126-10121  Troubleshoot  the  Ml/MlAl  lank  using  drivers  control  panel  warning  and  caution  lights 

Task  3:  Vehicle  maintenance  ( 171-126-10171  Perform  be forc-ope rations  checks  and  scivices  on  an  Ml/MlAl  tank 

Task  4:  \'ehicle  maintenance  1 171-126-10181  Perform  du ring-operations  checks  and  services  on  an  Ml/MLAl  lank 

Task  5:  N'chicle  maintenance  1 171-126-10191  Perform  aiier-opcrations  checks  and  services  on  an  Ml/MlAl  tank 

Task  6:  Maintain  operator  s  pan  of  equipment  record  folder 

Sutkxi  3-Lai>d  Nav/First  Aid 
Task  1:  Identifv'  terrain  features  on  a  map 

Task  2;  Determine  the  grid  coordinates  of  a  point  on  a  military  map  using  the  military  grid  reference  system 

Task  3;  Splint  a  suspected  fracture 

Task  4:  Pul  on  a  field  or  pressure  dressing 

Statioo  4-Lxjader’s  Statioa 

Task  1:  Prepare  loader’s  station  for  operation  on  an  Ml/MlAl  tank  (171-126-1023) 

Task  2:  Prepare  loader’s  station  for  operation  on  an  Ml/MlAl  tank  (171-122-1017)  Install/rcmove  an  M240  loader’s  machine  gun  on 

an  Ml/MlAl  tank 

Task  3:  Clear  and  load  an  M240  machine  gun 

Task  4:  Load  and  unload  the  105mm  mam  gun  on  an  Ml/MlAl  tank 

Task 5;  Secure  loader’s  station  on  an  Ml/MlAl  tank  (171-126-10241 

Task  6:  Secure  loader's  station  on  an  Ml/MlAl  tank  (171-122-1017)  Install/rcmove  an  M240  loaders  machine  gun  on  an  Ml/MlAl 

tank 

Task  7:  Clear  a  ^0  cal  M2  HB  machine  gun  to  prevent  accidental  discharge 
Task  8:  Inspect  120mm  ammunition  for  serviceability 

Statioa  5-Nuc/Biol/Cbem 

Task  1:  Put  on,  wear,  remove  and  store  an  M25A1  protective  mask  with  hood 
Task  2;  Recognize  and  react  to  chemical  or  biological  hazard 
Task  3:  React  to  nuclear  hazard 

Task  4:  Decontaminate  your  skin  and  personal  equipment 

Note:  The  school  administers  cither  tasks  5  and  6  or  7  amd  8  from  station  5  to  the  soldiers.  However,  the  school  periodically 
administers  all  four  usks  to  the  soldiers.  The  school  indicated  that  equipment  availability,  weather  conditions  and  instructor’s 
discretion  determine  who  received  which  tasks. 

Task  5:  Administer  nerve  agent  antidote  to  self  (self-aid) 

Task  6:  Use  MS  detector  paper  to  detect  chemical  agent 
Task  7:  Administer  first  aid  to  a  nerve  agent  casually  (buddy-aid) 

Task  8:  Use  M9  detector  paper  to  detect  chemical  agent 
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Table  11! 


19K  Armor  Stakes  Version  B 


Task  Station 


SutioQ  l-Nuc/Biol/Qikeixi 

Task  1:  Put  on.  wear,  remove,  and  store  your  M25.  M25A1  protective  mask  with  hood 
Task  2:  Recognize  and  react  to  chemical  or  biological  hazard 
Task  3:  React  to  nuclear  hazard 

Task  4:  Decontaminate  your  skin  and  personal  equipment 

Note:  The  school  administers  cither  tasks  5  .and  6  or  tasks  7  and  8  from  station  1  to  most  of  the  soldiers.  However,  the  school 
periodically  administers  all  four  tasks  to  the  soldiers.  The  school  indicated  that  equipment  availability,  weather  conditions  and 
instructor’s  discretion  determine  who  receives  which  tasks. 

Task  5:  Administer  nerve  agent  antidote  to  self  (self-aid) 

Task  6:  Use  M-8  detection  paper  to  ID  chemical  agents 

Task  7:  Administer  first  aid  to  a  nerve  agent  casualty  (buddy-aid) 

Task  &  Use  M-9  detection  paper  to  ID  chemical  agents 

Sutioo  Z-Maintenance 

Task  1:  Extinguish  a  fire  on  an  Ml/MlAl  tank 

Task  2:  Troubleshoot  the  Ml/MlAl  tank  using  drivers  instrument  control  panel,  warning  and  caution  lights 

Task  3:  Perform  be  fore -operations  checks  and  services  on  an  Ml/MlAl  tank 

Task  4:  Perform  during-opcrations  checks  and  service  on  an  Ml/MlAl  tank 

Task  5:  Perform  after-operations  checks  and  service  on  an  Ml/MlAl  tank 

Task  6;  .Maintain  operators  pan  of  equipment  record  folder 

Sutioo  3-Laod  Nav/Fust  Aid 
Task  1:  ID  terrain  features  on  a  map 
Task  2:  Determine  grid  ccxirdinates 

The  school  administered  task  3  to  some  soldiers,  in  addition  to  tasks  1.  2,  4.  and  5  from  station  3.  The  school  indicated  that 
equipment  availability,  weather  conditions  and  instructor's  discretion  determine  who  receives  which  tasks. 

Task  3:  Estimate  range 

Task  4:  Splint  a  suspected  fracture 

Task  5:  Put  on  a  field  pressure  dressing 

Sutioo  4-Loader'S  Sutioo 

Task  1;  Inspect  120mm  ammunition  for  serviceability 

Task  2:  Prepare  loader's  station  for  operations  on  an  Ml/MlAl  tank 

Task  3:  Install  and  remove  loader’s  M240  machine  gun  on  an  Ml/MlAl  tank 

Task  4:  Clear  and  load  an  M240  machine  gun 

Task  5:  Load  and  unload  the  105mm  main  gun  on  an  Ml/MlAl  tank 
Tuk  6:  Secure  loader's  station  on  an  Ml/MlAl  tank 

The  school  administered  task  7  from  station  4  to  most  soldiers.  The  school  indicated  that  equipment  availability,  weather 
conditions  and  instructor’s  discretion  determine  who  receives  which  tasks. 

Task  7:  Install  and  remove  loader’s  M240  machine  gun  on  an  Ml/MlAl  tank 
Task  8:  Clear  a  .50  cal  M2  HB  machine  gun 

SutioQ  S-WeapOQs/Cooimunicatiofis 
Tadt  1:  Maintain  an  M9  pistol 

Task  2:  Load,  reduce  stoppage  and  clear  an  M16A1/A2  rifle 
Task  3:  Recognize  friendly  and  threat  armored  vehicle  and  aircraft 
Task  4:  Prepare/operate  FM  radio  set 
Task  5:  Operate  intercom  set  AN/VRC-1  on  a  tracked  vehicle 
Task  6:  Send  a  radio  message 
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Tabic  112 


19K  Armor  Stakes  Version  C 


Task  Station 


SUtioD  1-Nuc/Biol/Cbcm 

Task  1:  Put  on.  wear,  remove  and  store  M25  protective  mask  with  hood 
Task  2:  Recognize  and  react  to  chemical  or  biological  hazard 
Task  3:  React  to  a  nuclear  hazard 

Task  4:  Decontaminate  your  skin  and  personal  equipment 
Task  5:  Use  M-8  detector  paper  to  ID  chemical  agent 
Task  6:  Administer  nerve  agent  antidote  to  self  (self-aid) 

Task  7:  Administer  first  aid  to  a  nerve  agent  casualty  (buddy-aid) 

Task  8:  Use  M9  paper  to  detect  chemical  agent 

Sutkxi  2-MainteDance 

Task  1:  Extinguish  a  fire  on  an  Ml/MlAl  series  tank 

Task  2:  Troubleshoot  the  Ml/MlAl  lank  using  drivers  instrument  control  panel  \fc-aming  and  caution  lights 

Task  3;  Perform  bcforc-operations  checks  and  services  on  an  Ml/MlAl  lank 

Task  4:  Perform  during-operations  checks  and  services  on  an  Ml/MlAl  tank 

Task  5:  Perform  after-operations  checks  and  services  on  an  Ml/MlAl  lank 

Task  6:  Maintain  operators  pan  of  equipment  record  folder 

Sutkxi  3-Laiid  Nav/First  Aid 
Task  1:  Identify  terrain  features  on  a  map 
Task  Z  Determine  the  grid  coordinates 
Task  3:  Splint  a  suspected  fracture 
Task  4:  Put  on  a  field  or  pressure  dressing 

Sutkxi  4-Loader’S  Sutkxi 
Task  1:  Prepare  loader’s  station  for  operations 
Task  Z  Insull/rcmovc  loader’s  M240  machine  gun 
Task  3:  Clear/load  M240  machine  gun 
Task  4:  Load /unload  105mm  main  gun 

Task  5:  Secure  loader’s  station 

Task  6:  Inspect  120mm  ammunition 

Task  7:  Clear  M2  HB  machine  gun 

Sutkm  S-Weapoos/Communicatkxis 

Task  1:  Maintain  an  M9  pistol 

Task  Z  Unload  and  M16A1/M16A2  rifle 

Task  3:  Recognize  friendly  and  threat  armored  vehicles  and  aircraft 
Task  4:  Prepare  radio  set  AN/VRC-64  or  AN/GRC-160  for  operation 

Task  5:  Operate  radio  set  AN/VRC-64  or  AN/GRC-160 

Task  6:  Operate  intercom  set  AN/VRC-1  on  a  tracked  vehicle 
Task  7;  Send  a  radio  message 


Tabic  113 

19K  Amor  Stakes  Version  D 


Task  Station 


Sutioa  1-Nuc/Biol/Chcm 

T«k  1:  Put  on.  wear,  remove  and  store  M25/M25A1 
Task  2:  Reco^ize  and  react  to  chemical  or  biological  hazard 
Task  3;  React  to  a  nuclear  hazard 

Task  4:  Decontaminate  your  skin  and  personal  equipment 
Task  5:  Administer  nerve  agent  antidote  to  self  (self-aid) 

Task  6:  Use  M8  paper  to  identify  chemical  agent 

Task  7:  Administer  first  aid  to  a  nerve  agent  casualty  (buddy-aid) 

Task  8;  Use  M9  paper  to  detect  chemical  agent 

Statioo  2-Maiateiiaace 

Task  1:  Extinguish  a  fire  on  an  Ml/MlAl  scries  tank 

Task  2:  Perform  before,  during,  and  after-operation  checks  and  services  on  an  Ml/MlAl  series  tank 
Task  3:  Maintain  operators  part  of  equipment  record  folder 

Task  4:  Troubleshoot  the  Ml/.MlAl  tank  using  drivers  control  panel  warning  and  caution  lights 

SutioQ  3-LaDd  Nav/Fust  Aid 
Task  1:  Identify  terrain  features  on  a  map 

Task  2:  Determine  the  grid  coordinates  of  a  point  on  a  military  map 

Task  3:  Splint  a  suspected  fracture 

Task  4:  Put  on  a  field  or  pressure  dressing 


Statioo  4-Loadcr's  Statioo 
Task  1:  Prcparc/opcratc  FM  radio  sets 
Task  2:  Send  a  radio  message 

Task  3:  Prepare  loader’s  station  for  operation  on  an  Ml/MlAl  scries  lank 
Task  4:  Clear  and  load  an  M240  machine  gun 

Task  5:  Load/unload  the  105mm  main  gun  on  an  Ml/MlAl  scries  tank 

Task  6:  Secure  loader's  station  on  an  Ml  /MlAl  tank 

Task  7:  Inspect  120mm  ammunition  for  serviceability 

Task  8:  Communicate  using  visual  signaling  techniques  mounted 

Sutioa  S-Weapoos 
Task  1:  Maintain  an  M9  pistol 

Task  2:  Clear  cal  ^0  M9  machine  gun  to  prevent  accidental  discharge 
Task  3:  Clear  an  M16A1/A2  rifle 

Task  4:  Recognize  friendly  and  threat  armored  vehicles  and  aircraft 
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Table  114 


IQK  Armor  Stakes  Version  E 


Task  Station 


Statkxi  l-Nuc/Bid/Chem 

Task  1:  Put  on.  wear,  remove  and  store  M25/M25A1 
Task  2:  Recognize  and  React  to  chemical  or  biological  hazard 
Task  3:  Decontaminate  your  skin  and  personal  equipment 
Task  4:  Administer  nerve  agent  antidote  to  self  (self  aid) 

Task  5:  Use  M8  paper  to  identify  chemical  agent 

Task  6:  Administer  first  aid  to  a  nerve  agent  casualty  (buddy  aid) 

Task  7:  Use  M9  paper  to  detect  chemical  agent 

Sutkm  2-Maiiitefia2Ke 

Task  1:  Extinguish  a  Tier  on  an  Ml/MlAl  tank 

Task  2:  Perform  before-operation  checks  and  services  on  an  Ml/MlAl  tank 
Task  3:  Perform  du  ring-opera  lions  checks  and  serMccs  on  an  Ml/MlAl  tank 
Task  4:  Perform  after-operations  checks  and  seiviccs  on  an  Ml/MlAl  tank 
Task  5:  Maintain  operators  part  of  equipment  record  folder 

Sutioa  3-Lafid  Nav/First  Aid 
Task  1:  Identify  terrain  features  on  a  map 

Task  2;  Determine  the  gnd  coordinates  of  a  point  on  a  military  map 

Task  3:  Splint  a  suspected  fracture 

Task  4:  Put  on  a  field  or  pressure  dresing 

Sutioa  4-Loader'S  Sutioa 
Task  1:  Prepare/operate  FM  radio  sets 
Task  2:  Send  a  radio  message 

Task  3:  Prepare  loader's  station  for  operation  on  an  Ml/MlAl  series  tank 
Task  4;  Clear  and  load  and  M240  machine  gun 

Task  5:  Load/unload  the  105mm  main  gun  on  an  Ml/MlAl  scries  tank 

Task  6:  Secure  loader’s  station  on  an  Ml/MlAl  tank 

Task  7:  Inspect  120mm  ammunition  for  serviceability 

Task  &  Communicate  using  visual  signaling  techniques  mounted 

Sutioo  5-Weapoos 
Task  1:  Maintain  an  M9  pistol 

Task  2;  Clear  cal  SO  M2  machine  gun  to  prevent  accidental  discharge 
Task  3:  Clear  an  M16A1/A2  rifle 

Task  4:  Recognize  friendly  and  threat  armored  vehicles  and  aircraft 


had 

2: 


Sample.  The  total  19K  sample  consisted  of  1,529 
the  following  numbers  of  subjects:  Version  A  Form 
26;  Version  B:  390;  Version  C:  120;  Version  0: 


subjects.  The  Versions 
1:  869;  Version  A  Form 
121;  Version  E:  3. 
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Results  and  Discussion.  We  recoded  each  score  so  that  soldiers  who  passed 
a  task  on  the  first  trial  received  a  score  of  2;  those  who  passed  on  the  second 
trial  received  a  score  of  1;  and  those  who  failed  the  task  received  a  score  of 
0.  Initially,  we  factor  analyzed  the  19K  variables  at  the  task  (item)  level 
separately  for  Version  A  Form  1,  and  Versions  B,  C,  and  D,  and  we  found 
strikingly  different  results  for  each  analysis.  Version  A  Form  1  (with  800 
cases)  demonstrated  no  clear  factor  pattern  at  all,  with  low  communal ities  (range 
from  .000  to  .633,  depending  upon  the  number  of  factors  extracted).  Version  B, 
with  limited  items  from  Stations  1,  4,  and  5,  showed  a  tendency  for  tasks  within 
stations  to  group  together.  We  were  unable  to  factor  analyze  Version  C,  due  to 
negative  eigenvalues.  Version  D  displayed  no  consistent  or  discernible  pattern. 

For  the  above  reasons,  we  decided  to  group  tasks  by  summing  into  logical 
content  groupings,  namely  the  stations.  We  also  created  a  Communications 
category,  because  such  items  had  appeared  in  either  Weapons/Communications  for 
Versions  A,  B,  and  C,  or  in  Loader’s  Station  for  Version  D.  We  did  not  separate 
Land  Navigation/First  Aid  into  two  categories,  since  this  grouping  contained  only 
four  or  five  items,  too  small  a  number  from  which  to  develop  two  scales.  This 
procedure  resulted  in  a  total  of  six  categories.  Further,  we  prorated  scores  by 
computing  averages  of  available  scores  within  a  category  so  as  to  minimize  the 
deletion  of  cases  with  missing  data.  As  an  indication  of  the  extent  of  missing 
data,  we  computed  a  missing  data  function  on  the  tasks  which  made  up  each 
category.  Out  of  the  total  of  24  categories,  17  had  some  missing  data.  Over  the 
4  versions  by  category,  the  percentages  of  cases  with  missing  data  ranged  from 
.1%  to  40.3%,  with  a  median  of  .8%.  The  percentages  of  cases  missing  more  than 
50%  of  the  tasks  within  a  category  ranged  from  .2%  to  40.3%,  with  a  median  of 
.8%.  These  small  percentages  suggest  no  systematic  pattern  to  the  missing  data, 


175 


but  given  our  prorating  (or  imputation)  procedure,  we  could  not  investigate  this 
possibility.  Other  researchers  recommend  other  rules  for  handling  missing  data, 
such  as  a  50%  rule,  in  which  a  case  is  excluded  if  the  subject  has  fewer  than 
half  of  the  items  which  make  up  the  scale. 

Table  115  displays  descriptive  statistics  on  the  aggregated  variables  for 
Version  A  Form  1,  and  Versions  B,  C,  and  D.  We  did  not  analyze  Version  A  Form 
2  and  Version  E,  due  to  the  small  number  of  cases  in  each  (26  and  3, 
respectively) . 

We  computed  the  mean  of  the  six  composite  variables  computed  above  as  a 
summary  measure  (Mean  in  Table  115),  with  descriptive  statistics  also  shown  in 
Table  115.  We  did  not  prorate  this  average  score.  This  procedure  resulted  in 
excluding  a  case  from  the  average  if  the  case  had  a  missing  composite  variable. 
Other  researchers  might  want  to  prorate  the  average  scores  in  addition  to  the 
composite  variables.  Researchers  could  also  correlate  the  missing  data  function 
described  above  to  the  average  scores  to  study  the  relationship  of  missing  data 
to  examinee  performance. 

Because  the  factor  analyses  differed  across  versions,  we  decided  to  conduct 
further  tests  to  determine  whether  the  soldiers  in  the  various  versions  differed 
(i.e.,  came  from  different  populations).  First,  we  conducted  multivariate 
analysis  of  variance  (MANOVA)  on  the  six  composite  scores.  We  found  that  the 
covariance  matrices  differed  across  the  four  versions  (F=22.24,  p  <  .001),  as  did 
the  means  (F=50.05,  p  <  .001).  Univariate  F  tests  indicated  that  all  six 
composite  variables  differed  across  versions. 
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Table  115 

Descnmive  Statistics  tor  19K  Composne  Variables 


Composuc 

JT 

Rel* 

Mean 

Sid  Dev 

Skewness 

Kurtosis 

Minimum 

Maximum 

\  ersion  A  Form  1 
Communicauons 

835 

.02 

1.967 

.150 

-4.891 

25-512 

.67 

2.00 

Weapons 

844 

.22 

1.760 

,293 

-1.099 

1.262 

.00 

2.00 

Land  Navigation /First  Aid 

S50 

.05 

1.914 

.209 

-2.508 

6,330 

.75 

2.00 

Loader’s  Station 

827 

.06 

1.783 

,212 

-.462 

-.972 

1.14 

2.00 

Maintenance 

847 

.08 

1,950 

.126 

-2.699 

8.975 

1.00 

2.00 

Nuc/Biol/Chem 

826 

.07 

1.856 

.204 

-1.418 

isn 

.67 

2.00 

Mean 

784 

.34 

1.873 

.098 

-1.081 

2.314 

1.33 

2.00 

Version  B 


Communications 

231 

.14 

1.921 

,292 

-4.670 

24.901 

.00 

2.00 

Weapons 

385 

.64 

1.799 

.354 

-1.931 

4.490 

.00 

2.00 

Land  Navigation /First  Aid 

232 

.01 

1.960 

.150 

-4.054 

17.436 

1.00 

2.00 

Loaders  Station 

390 

.04 

1.971 

.110 

-4.055 

17.326 

1.20 

2.00 

Maintenance 

385 

.12 

1.926 

.170 

-4.698 

42.905 

.00 

2.00 

Nuc/Biol/Chem 

388 

.04 

1.968 

.106 

-3.937 

20.686 

1.00 

2.00 

Mean 

224 

.45 

1.891 

.119 

-1.475 

3.993 

1.28 

2.00 

VcrsionC 


Communications 

117 

N/A 

2.000 

.000 

N/A 

N/A 

2.00 

2.00 

Weapons 

117 

.00 

1.977 

.122 

-5.194 

25.409 

1.33 

2.00 

Land  Navigation/First  Aid 

117 

.19 

1.949 

.152 

-2.654 

5.132 

130 

2.00 

Loaders  Station 

117 

.16 

1.991 

.065 

-7348 

55.930 

130 

2.00 

Maintenance 

119 

N/A 

1.997 

.031 

-10.909 

119.000 

1.67 

2.00 

Nuc/Biol/Chem 

119 

.24 

1.978 

.081 

-3.839 

15.050 

130 

2.00 

Mean 

117 

.35 

1.983 

.041 

-2.650 

7,120 

1.79 

2.00 

Version  D 

Communications 

95 

.N/A 

2.000 

.000 

N/A 

N/A 

2.00 

2.00 

Weapons 

120 

J3 

li58 

.450 

-.867 

.058 

30 

2.00 

Land  Navi  gat  ion /First  Aid 

120 

.06 

1.554 

.409 

-1.047 

1-594 

.00 

2.00 

Loader’s  Station 

118 

.10 

1.983 

.081 

-4.602 

19312 

1.60 

2.00 

Maintenance 

120 

N/A 

2.000 

.000 

N/A 

N/A 

2.00 

2.00 

Nuc/Biol/Chem 

120 

.02 

1.892 

.208 

-1.960 

3.447 

1.00 

2.00 

.Mean 

93 

.40 

1.825 

.121 

-.765 

-.073 

130 

2.00 

Note.  The  variable  "Mean"  is  (Communications  +  Weapons  +  Land  Navigation/First  Aid  +  Loader’s  Station  +  Maintenance  + 
Nuc/Biol/Chcm)/6 

*These  arc  composites  based  on  the  communality  estimates  and  the  procedures  described  in  the  text. 


To  explore  possible  reasons  for  the  observed  differences  between  versions, 
we  attempted  to  examine  ability  as  a  cause.  Since  we  had  the  soldiers’  pre¬ 
enlistment  ASVAB  scores,  we  could  use  the  ASVAB  as  a  measure  of  ability  and 
statistically  adjust  the  composite  and  control  for  differences  in  ability. 
(Incidentally,  in  comparing  the  different  versions  (Al,  B,  C,  and  D)  on  the  10 
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ASVAB  scores  using  MANOVA.  the  ASVAB  correlation  matrices  did  not  differ  across 
Versions  (F  =  .91;  p  =  .79,  ns),  but  the  10  ASVAB  means  did  differ  across 
versions  (approximate  F  =  1.53;  p  =  .03).) 

To  partial  out  ability  from  the  composite  variables,  we  performed 
multivariate  analysis  of  covariance  on  the  six  composite  measures  using  the  10 
ASVAB  scores  as  covariates.  The  test  for  homogeneity  of  dispersion  matrices 
indicated  that  the  correlation  matrices  differed  across  versions  (F  =  4.35,  p  < 
.001).  Generally,  Version  A  had  larger  variances  and  covariances  than  the  other 
versions,  and  since  Version  A  had  far  more  cases  than  any  of  the  other  versions, 
this  finding  produces  a  conservative  alpha  level  for  the  test  of  means 
(Tabachnick  &  Fidell,  1983,  p.  233).  The  means  for  the  six  composite  scores  did 
differ  across  the  four  Versions  (approximate  F  =  49.86;  p  <  .001).  Univariate 
F-tests  indicated  that  the  means  of  all  six  composite  variables  differed 
significantly  across  versions.  Of  35  individual  pairwise  comparisons  among  the 
means,  29  differed  between  versions  at  the  p<.05  level. 

These  results  indicate  that,  after  controlling  for  ability,  the  soldiers 
differed  in  their  performance  on  the  composite  tasks,  both  in  level  and  pattern. 
These  results  implicate  something  other  than  soldiers’  ability  in  producing  the 
observed  differences  between  versions  (e.g.,  through  an  order  effect),  but  we  do 
not  have  enough  information  to  fully  answer  this  question.  This  suggests 
researchers  should  not  combine  scores  from  different  versions. 

However,  to  further  evaluate  the  conclusion  of  differences  between 
versions,  we  computed  effect  sizes  as  measures  of  the  practical  significance  of 
the  differences  between  the  unadjusted  means  across  versions.  For  example, 
comparing  Versions  A  and  B  on  the  Communications  variable  =  1.967,  Mg  = 
1.921),  the  effect  size  was  .24.  Cohen  and  Cohen  (1977)  say  that  .2  represents 
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a  small  effect  size,  .5  is  medium,  and  .8  is  large.  For  the  total  of  35  possible 
comparisons  among  the  19K  versions,  we  found  the  following  frequencies  within 
these  effect  size  categories:  .00-. 19  (9),  .20-. 49  (10),  .50-. 79  (8),  and  .80+ 
(8).  These  results  show  that  54%  of  the  effect  sizes  fall  below  a  value  of  .50. 
Thus,  while  the  MANOVA  tests  indicate  that  the  versions  differ  from  one  another 
in  terms  of  statistical  significance,  the  effect  sizes  suggest  small  magnitudes 
of  the  differences,  in  terms  of  practical  significance. 

To  further  attempt  to  obtain  a  useful  summary  measure  of  soldier 
performance,  we  submitted  the  six  composite  variables  to  factor  analysis, 
separately  for  each  version.  Table  116  presents  the  commonalities,  factor 
loadings,  and  factor  score  coefficients  for  each  version.  We  found  a  one-factor 
solution  in  each  version.  Table  117  displays  correlations  among  the  six 
composite  scores,  the  mean  composite  score,  and  the  factor  score  for  each 
version.  The  separate  composite  measures  do  not  relate  highly  to  one  another, 
though  they  correlate  more  highly  with  the  mean  composite  measure.  Table  117 
also  shows  the  correlations  between  the  factor  score  and  each  composite  score  and 
the  mean  composite  score.  The  correlations  of  the  factor  score  with  the 
composites  ranged  from  .02  to  .97,  while  the  correlations  of  the  factor  score 
with  the  mean  composite  ranged  from  .81  to  .97. 
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Table  116 

C'ommunaiuies.  Facior  Loadings,  and  Factor  Score  Coefficienis  lor  19K  Cbmposiie  Variables 


Composite 

Communaliiy 

Factor  Loading 

Factor  Score  Coefficient 

\'ersion  .A  Form  1 

Communications 

.025 

.159 

.106 

Weapons 

.220 

.469 

.377 

Land  Navigation/First  Aid 

.052 

.229 

.152 

Loader’s  Station 

.063 

.250 

.167 

Maintenance 

.084 

.289 

.197 

Nuc/Biol/Chcm 

.068 

.260 

.174 

Version  B 

Communications 

.139 

.373 

.122 

Weapons 

.644 

.803 

.711 

Land  Navigation /First  Aid 

.012 

.109 

.027 

Loader  s  Station 

.(MO 

.201 

.093 

Maintenance 

.115 

.338 

.120 

Nuc/Bio!/Chem 

.037 

.192 

.046 

\'crsion  C 

Communications 

N/A 

S/A 

S/A 

Weapons 

.000 

.013 

.008 

Land  Navigation/First  Aid 

.193 

.439 

.311 

Loader's  Station 

.160 

.400 

.272 

Maintenance 

S/A 

S/A 

N/A 

Nuc/Biol/Chem 

.243 

.493 

.371 

Version  D 

Communications 

N7A 

N/A 

N/A 

Weapons 

.527 

.726 

.666 

Land  Navigaiion/Firsi  Aid 

.056 

.238 

.113 

Loader’s  Station 

.101 

.318 

.142 

Maintenance 

S/A 

N/A 

N/A 

Nuc/Biol/Chcm 

.019 

.138 

.138 

In  summary,  the  limitations  of  the  raw  data  include  their  essential 
dichotomous  (pass/fail)  nature  at  both  the  item  level  and  at  the  overall 
performance  level.  At  the  item  level,  even  measuring  number  of  trials  to 
passing,  the  variability  was  so  limited  that  for  all  practical  purposes,  almost 
all  students  pass  on  the  first  trial.  A  student  either  graduates  or  fails  to 
graduate,  with  no  other  recorded  measure  of  overall  school  performance.  However, 
the  current  research  indicates  that  obtaining  separate  station  scores  to  yield 
one  summary  measure  may  hold  some  degree  of  promise  for  validation  research. 
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Table  IIT 

Correlations  Among  ihe  Composite  Variables  in  19K  Versions 


Composite 

Comm 

Weap 

LN/FA 

Load 

.Maim 

N/B/C 

Mean 

V  ersion  A  Form  1  TN  =  784) 
Weapons 

.03 

Land  Navigation /First  Aid 

.08 

.10 

Loader's  Station 

.14 

.13 

.04 

Maintenance 

.05 

.14 

.07 

.04 

Nuc/Bioi/Chem 

..01 

.14 

.06 

.04 

.09 

Mean 

.36 

.65 

.47 

.50 

.37 

.47 

Factor 

.26 

.77 

.38 

.41 

.47 

.43 

.97 

Version  B  fN  =  224^ 
Weapons 

.31 

Land  Navi  gat  ion /First  Aid 

-.06 

.11 

Loader  s  Station 

.21 

.10 

-.01 

Maintenance 

.11 

.27 

.05 

.10 

.\uc/Biol/Chcm 

-.03 

.19 

.13 

-.01 

.07 

Mean 

.60 

.82 

.28 

.36 

.52 

.29 

Factor 

.45 

.97 

.13 

.24 

.41 

.23 

.93 

Version  C  fN  =  117) 
Weapons 

N/A 

La.  J  Navigation/First  Aid 

S/A 

.09 

Loaders  Station 

N/A 

-.02 

.17 

Maintenance 

N/A 

N/A 

N/A 

N/A 

Nuc/Biol/Chem  j 

N/A 

-.05 

.22 

.20 

N/A 

Mean 

N/A 

.78 

.42 

N/A 

.46 

Factor 

N/A 

.02 

.67 

.61 

N/A 

.75 

.81 

Version  D  fN  *  93) 

Weapons 

N/A 

Laud  Navigation/First  Aid 

N/A 

.10 

Loader's  Station 

N/A 

.28 

.18 

Maintenance 

N/A 

N/A 

N/A 

N/A 

Nuc/Biol/Chcm 

N/A 

.09 

.21 

-.09 

N/A 

Mean 

N/A 

.74 

.68 

.34 

N/A 

.45 

Factor 

N/A 

.97 

.28 

.43 

N/A 

.18 

.86 

short  of  redesigning  the  19K  tests  to  have  more  desirable  psychometric 
properties.  Researchers  also  need  to  investigate  differences  among  the  versions. 

Recommendations.  Obtain  separate  station  scores  for  Communications, 
Weapons,  Land  Navigation/First  Aid,  Loader’s  Station,  Maintenance,  and 
Nuc/Biol/Chem,  and  obtain  the  mean  of  these  six  scores  to  yield  an  overall 
summary  measure.  Do  this  for  Version  A  Form  1  by  itself.  If  an  even  larger 
sample  size  is  desired,  combine  all  available  versions. 
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Air  Force  Schools 

Apprentice  Air  Traffic  Control  Operator  (27230) 

Description  of  Variables.  In  January  1991,  the  school  changed  the 
curriculum  from  five  blocks  to  six  blocks,  covering  the  same  material  but  in  a 
slightly  different  order.  Students  who  enrolled  prior  to  January  22,  1991 
received  the  old  curriculum;  those  who  enrolled  on  or  after  that  date  received 
the  present  curriculum.  The  school  broke  the  former  Block  I,  National  Air 
Traffic  Training  Program  (NATTP),  into  two  parts.  The  school  formed  the  Federal 
Aviation  Administration  (FAA)  computer-based  instruction  (CBI)  component  into 
Block  VI,  called  Federal  Control  Tower  Operator  (CTO)  Criteria.  Finally,  the 
school  renamed  the  remainder  of  Block  I  Air  Traffic  Control  Fundamentals. 

Student  training  hours  represent  the  number  of  hours  a  student  takes  to 
complete  a  block.  Blocks  III  &  V  each  include  two  training  sections  (a  &  b). 
The  school  also  changed  the  training  hours  for  the  blocks,  although  the  total 
hours  stayed  the  same  at  600: 

Block 

Curriculum  _I _ II  Ilia  Illb  IV  Va  Vb _ VI_  Total 

Former  155  55.5  77.5  40  73.5  40  158.5  —  600 

Current  81.5  56  78.5  40  73.5  48  149  73.5  600 

Students  frequently  spent  more  than  these  hours  in  training,  sometimes  up  to 
twice  as  many  hours  in  certain  Blocks. 

Table  118  describes  the  hours  and  test  variables  by  Block.  The  school 
administers  Written  Knowledge  Tests  for  Blocks  I,  II,  IV,  and  VI  (in  the  current 
curriculum);  Performance  Tests  for  Blocks  III  and  V;  and  Quizzes  for  Training 
Blocks  I,  II,  and  IV.  RGI  entered  test  grades  representing  the  scores  attained 
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Table  lib 

Descnptions  of  Air  Traffic  Control  Operator  (27230)  Variables 


Block 

Title 

Variables 

I 

National  Air  Traffic  Training  Program  (  former  cumculum) 

Hours.  Written  Test 

I 

Air  Traffic  Control  Fundamentals  fcurrcni  curriculum) 

Hours.  Written  Test 

II 

Control  Tower  Procedures 

Hours.  Written  Test 

Ilia 

Control  Tower  Operation:  Basic  Tower  Operation 

Hours.  Performance  Test 

Illb 

Control  Tower  Operation:  Advanced  Tower  Operation 

Hours.  Performance  Test 

I\' 

Radar  Approach  Control  Procedures 

Hours,  Written  Test 

Va 

Radar  Approach  Control  Operation:  Basic  Approach  Control  Operation 

Hours.  Performance  Test 

Radar  Approach  Control  Operation:  Advanced  Approach  Control  Operation 

Hours.  Performance  Test 

\T 

Federal  Control  Tower  Operator  Criteria  (current  curriculum) 

Hours.  Written  Test 

on  only  the  first  written  knowledge  tests  for  Training  Blocks  I,  II,  IV,  &  VI 
(for  the  current  curriculum).  Students  who  fail  the  exam  retake  it  after 
remediation.  For  those  passing  the  retest,  the  school  assigns  a  grade  of  70  and 
uses  it  in  calculating  the  Final  School  Grade  (FSG).  In  the  former  curriculum, 
the  school  calculated  FSG  as  the  mean  of  the  three  written  knowledge  tests.  In 
the  current  curriculum,  the  school  computes  FSG  as  the  mean  of  the  four  written 
knowledge  tests. 

Students  receive  performance  scores  for  Training  Blocks  III  and  V.  Since 
Blocks  III  and  V  each  include  two  training  sections,  students  receive  a  total  of 
four  performance  scores.  School  instructors  score  the  performance  tests  on  a 
Satisfactory  (S)  or  an  Unsatisfactory  (U)  basis.  For  research  purposes, 
researchers  scored  an  "S"  as  a  1  and  a  "U"  as  a  0. 

During  Training  Block  IV,  students  receive  quizzes  covering  specified 
course  objectives,  divided  into  9  Progress  Checks  (abbreviated  PCI  through  PC9). 
Students  receive  either  an  "S"  or  a  "U"  on  the  quizzes,  depending  on  whether  they 
meet  the  passing  score.  The  passing  score  (the  standard)  depends  upon  the  number 
of  items  per  quiz: 


183 


Number  of  items  on  quiz:  3  4  5  6  7  8  9  10  11  12  13  14  15 

Standard:  2  3  3  4  5  5  6  7  8  9  9  10  11. 

Students  who  fail  a  quiz  retake  it  until  they  pass.  For  research  purposes, 
we  entered  the  sum  of  the  "U"s  on  the  initial  quizzes  for  Training  Block  IV  from 
the  school’s  Summary  Data  Sheets  (Form  667)  into  the  database.  This  summary 
variable  was  Radar  Approach  Control  Procedures  (Block  IV),  with  a  valid  range  of 
digits  from  0-29. 

We  also  entered  scores  for  Block  IV  quizzes  (the  number  of  correct 
responses  per  quiz).  The  school  changed  the  curriculum  during  data  collection. 
For  this  reason,  the  number  of  quizzes  and  the  number  of  items  per  quiz  differs, 
tudents  in  class  numbers  through  June  21,  1990  received  up  to  32  quizzes  labeled 
la  through  7c  (Curriculum  1).  Students  in  class  numbers  from  June  22,  1990 
through  January  21,  1991  received  up  to  29  quizzes  labeled  la  through  lad 
(Curriculum  2).  Students  with  class  numbers  from  January  22,  1991  through 
September  29,  1991  received  23  quizzes  labeled  la  through  Iw  (Curriculum  3). 
Students  in  class  numbers  from  September  30,  1991  to  the  present  receive  24 
quizzes  labeled  la  through  lx  (Curriculum  4).  However,  Summary  Data  Sheets  for 
both  Curricula  3  and  4  list  the  quizzes  as  la  through  Iw.  The  school  continued 
to  use  old  summary  sheets  after  September  29,  1991.  For  defining  the  class  dates 
researchers  used  the  entry  date  in  the  upper  left-hand  corner  of  the  Form  156 
instead  of  the  current  class  number  located  in  the  upper-right  hand  corner  of 
that  form. 

The  school  cannot  provide  specific  Block  IV  quiz  content  for  classes  prior 
to  900625  (Curriculum  1).  The  school  labeled  the  quizzes  la  through  7c  on  the 
Summary  Data  Sheets.  However,  researchers  renamed  the  quizzes  and  entered  them 
as  they  appeared  on  the  Performance  Checklists  because  they  could  not  determine 
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the  specific  quiz  content.  The  actual  number  of  quizzes  that  the  school 
admini stored  may  not  match  the  number  of  quizzes  specified  in  the  summary  sheet. 
The  following  list  provides  the  number  of  items  per  quiz  for  Block  IV,  Curriculum 
1: 

PCI  Quiz  1  (7),  PC2  Quiz  1  (11),  PC2  Quiz  2  (7),  PC2  Quiz  3  (10), 

PC3  Quiz  1  (4),  PC3  Quiz  2  (10),  PC3  Quiz  3  (10), 

PC4  Quiz  1  (10),  PC4  Quiz  2  (10),  PC4  Quiz  3  (6), 

PCS  Quiz  1  (4),  PCS  Quiz  2  (5),  PCS  Quiz  3  (10), 

PCS  Quiz  1  (9),  PCS  Quiz  2  (4),  PCS  Quiz  3  (6), 

PC7  Quiz  1  (10),  PC7  Quiz  2  (4),  PC7  Quiz  3  (10),  PC7  Quiz  4  (4), 

PCS  Quiz  1  (10),  PCS  Quiz  2  (4),  PCS  Quiz  3  (4),  PCS  Quiz  4  (10), 

PCS  Quiz  S  (4),  PC9  Quiz  1  (7),  PC9  Quiz  2  (4),  and  PC9  Quiz  3  (10). 

The  school  cannot  provide  specific  Block  IV  quiz  content  for  several 
quizzes  used  in  classes  900628  through  910121  (Curriculum  2).  The  school  labeled 
the  quizzes  la-lac  on  the  Summary  Data  Sheet.  However,  researchers  renamed  the 
quizzes  and  entered  them  as  they  appeared  on  the  Performance  Checklists  because 
the  researchers  could  not  determine  the  specific  quiz  content.  The  actual  number 
of  quizzes  that  the  school  administered  during  this  time  period  may  not  match  the 
number  of  quizzes  specified  on  the  summary  sheet.  The  following  list  provides 
the  number  of  items  per  quiz  for  Block  IV,  Curriculum  2: 

PCI  Quiz  1  (8),  PC2  Quiz  1  (5),  PC2  Quiz  2  (10),  PC2  Quiz  3  (9), 

PC3  Quiz  1  (4),  PC3  Quiz  2  (9),  PC3  Quiz  3  (10), 

PC4  Quiz  1  (9),  PC4  Quiz  2  (9),  PC4  Quiz  3  (4), 

PCS  Quiz  1  (3),  PCS  Quiz  2  (4),  PCS  Quiz  3  (4),  PCS  Quiz  4  (10), 

PCS  Quiz  1  (14),  PCS  Quiz  2  (6), 

PC7  Quiz  1  (6),  PC7  Quiz  2  (10),  PC7  Quiz  3  (4),  PC7  Quiz  4  (4), 
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PCS  Quiz  1  (10),  PCS  Quiz  2  (4),  PCS  Quiz  3  (8), 

PC9  Quiz  1  (4),  PC9  Quiz  2  (9),  and  PC9  Quiz  3  (5). 

The  Block  IV  quiz  information  presented  in  Table  119  represents  the 
curricula  effective  January  21,  1991  and  September  30,  1991  (Curricula  3  and  4). 
The  school  added  Quiz  If  into  the  curriculum  in  September  1991  and  administered 


Table  119 

Blcxrk  IV  Quizzes  bv  Progress  Checks  fPO  for  27230  Schcx)l 


PC  Quiz:  Content  (Number  of  Items  ) 


1  la  (Curriculum  4  only):  Identify  procedures  used  to  apply  IFR  separation.  (9) 

1  la  (Curriculum  3  only):  Identify  procedures  used  to  apply  IFR  separation.  (6) 

1  lb  (Curriculum  4  only):  Identify  procedures  used  to  control  IFR  departures.  (5) 

2  Ic  Identify  procedures  used  to  issue  holding  instructions.  (6) 

2  Id:  Identify  procedures  used  to  control  IFR  arrivals.  (9) 

3  Ic:  Identify  general  radar  procedures  applied  to  aircraft.  (5) 

3  If  (Curriculum  4  only):  Identify  general  principles  about  airpon  surveillance  radar  (ASR)  indicators.  (4) 

3  If  (Curriculum  3),  Ig  (Curriculum  4):  Identify  basic  facts  about  electronic  combat,  (5) 

3  Ig  (Curriculum  3),  Ih  (Curriculum  4):  Identify  procedures  used  to  assign  beacon  codes.  (10) 

4  Ih  (Curriculum  3),  li  (Curriculum  4):  Identify  procedures  used  to  establish  radar  identification.  (9) 

4  li  (Curriculum  3),  Ij  (Curriculum  4):  Identify  procedures  used  to  vcaor  aircraft.  (9) 

4  Ij  (Curriculum  3),  Uk  (Curriculum  4):  Identify  procedures  used  to  apply  radar  separation.  (11) 

5  Ik  (Curriculum  3),  11  (Curriculum  4):  Identify  procedures  used  to  issue  low  altitude  alerts.  (4) 

5  11  (Curriculum  3),  Im  (Curriculum  4):  Identify  procedures  used  to  provide  additional  services.  (8) 

5  Im  (Curriculum  3),  In  (Curriculum  4):  Identify  services  provided  under  terminal  radar  program.  (10) 

6  In  (Curriculum  3),  lo  (Curriculum  4):  Identify  procedures  used  to  control  radar  arrivals.  (14) 

6  lo  (Curriculum  3),  Ip  (Curriculum  4):  Identify  procedures  used  to  confirm  aircraft  altitudes.  (4) 

6  Ip  (Curriculum  3),  Iq  (Curriculum  4):  Identify  the  procedures  used  to  transfer  radar  identification  of  aircraft.  (12) 

7  Iq  (Curriculum  3),  Ir  (Curriculum  4):  Identify  procedures  used  to  control  radar  arrivals.  (15) 

7  It  (Curriculum  3),  Is  (Curriculum  4):  Identify  procedures  to  control  visual  approaches.  (4) 

7  Is  (Curriculum  3),  It  (Curriculum  4):  Identify  general  principles  about  contaa  approaches.  (4) 

8  It  (Curriculum  3),  lu  (Curriculum  4):  Identify  procedures  used  to  control  radar  approaches.  (5) 

8  lu  (Curriculum  3),  Iv  (Curriculum  4):  Identify  procedures  used  to  control  ASR  approaches.  (8) 

9  Iv  (Curriculum  3),  Iw  (Curriculum  4):  Identify  general  principles  about  precision  approach  radar  (PAR)  indicators.  (4) 

9  Iw  (Curriculum  3),  lx  (Curriculum  4):  Identify  procedures  used  to  control  PAR  approaches.  (8). 
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this  quiz  to  students  with  class  entry  date  numbers  from  910930  to  the  present. 
The  number  in  parentheses  following  each  quiz  description  gives  the  number  of 
items  on  the  quiz. 

Sample.  The  former  curriculum  contained  292  students,  while  the  current 
curriculum  contained  416  students,  for  a  total  of  708  students. 

Results  and  Discussion.  To  examine  the  comparability  of  the  two  curricula, 
we  tested  the  curricula  on  variables  common  to  both  by  using  MANOVA.  The 
variables  common  to  both  curricula  consisted  of:  hours  for  Blocks  2,  3A,  4,  5A, 
and  5b;  written  tests  for  Blocks  2  and  4;  and  performance  tests  for  Blocks  3A, 
5A,  and  5B.  The  MANOVA  indicated  significant  differences  between  the  covariance 
mcvrices  (F  =  5.130,  £<.001),  as  well  as  the  means  (F  =  11.742,  fi<.001). 
Univariate  F-tests  indicated  that  5  of  the  10  variables  differed  between 
curricula  at  the  £<.05  level. 

To  investigate  ability  as  a  possible  determinant  of  the  differences  between 
the  curricula,  we  conducted  MANCOVA  by  parti  ailing  out  ASVAB  pre-enlistment 
scores.  MANCOVA  indicated  that  the  covariance  matrices  still  differed  (F  = 
2.132,  £<.001),  as  well  as  the  means  (F  =  10.469,  £<.001).  Again,  univariate  F- 
tests  indicated  that  5  of  the  10  variables  differed  between  curricula  at  the 
£<.05  level.  For  these  reasons,  we  treated  the  former  and  current  curricula 
separately. 

Table  120  shows  descriptive  statistics  on  variables  in  the  •  former 
curriculum. 

Factor  analysis  of  the  former  curriculum  variables  (hours,  written  test 
scores,  performance  test  scores,  and  quiz  scores)  indicated  one  strong  factor, 
with  an  overall  five-factor  solution.  Table  121  shows  the  communalities  and 
factor  loadings  for  the  variables  in  the  factor  analysis,  while  Table  122  shows 
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factor  score  coefficients.  Factor  1  shows  a  written  test  or  FSG  factor,  although 
the  hours  for  Block  4  receives  a  negative  sign.  The  other  four  factors  contrast 
each  performance  test  with  the  corresponding  hours  variables  (i.e.,  test  3A  - 
hours  for  3A) .  The  results  suggest  that  each  performance  block  might  constitute 
an  independent  dimension,  also  indicated  by  their  low  correlations  with  the  other 
variables.  Because  the  performance  tests  had  a  ceiling  effect,  as  seen  in  Table 
120,  we  hypothesized  that  the  independence  of  the  performance  tests  might  have 
stemmed  from  a  statistical  artifact  of  the  ceiling  effect.  The  hours  variables 
were  not  perfectly  negatively  correlated  with  their  corresponding  block  tests. 
Thus,  we  also  desired  to  determine  if  hours  constituted  a  separate  dimension. 
Finally,  the  quizzes  did  not  appear  to  display  any  consistent  patterns  or 
relationships.  We  obtained  essentially  the  same  five-factor  solution  with  the 
quizzes  excluded  as  we  did  with  them  included.  Thus,  we  left  the  quizzes  in  the 
factor  scores,  but  since  the  quizzes  referred  only  to  Block  4,  we  did  not  develop 
composite  variables  from  the  quiz  scores. 
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Tabic  120 

Pescnruve  Statistics  for  27230  \'ariahies  >  Former  Curriculum 


\‘anable“  r\  Mean  Std  Dev  Skewness  Kunosis  Minimum  Maximum 


Block  1  (Nat,  Air  Traffic  Training  Prog.)  Hours 

292 

158.863 

21.599 

.283 

6.638 

48.0 

264.0 

Block  2  (Control  Towtr  Proc.)  Hours 

270 

60.838 

15.624 

2.036 

2.981 

40.5 

119i 

Block  3A  (Basic  Tower  Op.)  Hour 

258 

86.492 

21.891 

1J37 

2,766 

22.8 

183J 

Block  3B  (Advanced  Tower  Op.)  Hour 

240 

.39,385 

4.881 

3,150 

25.114 

24.0 

72.0 

Block  4  (Radar  Approach  Control  Proc.)  Hour 

239 

80.061 

23.728 

2.637 

6.736 

57,5 

2143 

Block  5A  (Basic  Approach  Control  Op.)  Hour 

231 

51,417 

13.060 

2.655 

6.390 

16.0 

96.0 

Block  5B  (Advanced  Approach  Control  Op.)  Hour 

226 

150.265 

19.699 

1.796 

8,457 

74.5 

2683 

Block  1  (Nat.  Air  Traffic  Training  Prog.)  Written  Test 

283 

78.972 

8.025 

-  .348 

-.129 

52.0 

97.0 

Block  2  (Control  Tower  Proc.)  Written  Test 

270 

80.841 

11.472 

-  .759 

.687 

38.0 

100.0 

Block  4  (Radar  Approach  Control  Proc.)  Written  Test 

239 

80.075 

10.346 

-  .722 

.614 

44.0 

100.0 

Block  3A  (Basic  Tower  Op.)  Perf.  Test 

245 

.914 

.281 

-2.978 

6.925 

0.0 

1.0 

Block  3B  (Advanced  Tower  Op.)  Perf.  Test 

240 

.983 

.128 

-7.599 

56.206 

0.0 

1.0 

Block  5A  (Basic  Approach  Control  Op.)  Perf.  Test 

228 

.873 

.334 

-2.253 

3.102 

0.0 

1.0 

Block  5B  (Advanced  Approach  Control  Op.)  Perf.  Test 

227 

.872 

.335 

-2.245 

3.068 

0.0 

1.0 

Proeress  Check  1  Quiz  1 

177 

7.209 

.896 

-  .953 

.300 

4.0 

8.0 

Progress  Check  2  Quiz  1 

177 

4.486 

.700 

-1.408 

2.613 

1.0 

5.0 

Progress  Check  2  Quiz  2 

177 

S.859 

1.181 

-1.020 

.666 

5.0 

10.0 

Progress  Check  2  Quiz  3 

176 

7.682 

1.243 

-  .712 

-  .400 

4.0 

9.0 

Progress  Check  3  Quiz  1 

177 

3.706 

.587 

-2.042 

3.795 

1.0 

4.0 

Progress  Check  3  Quiz  2 

177 

7.910 

.955 

-.609 

-  .334 

5.0 

9.0 

Progress  Check  3  Quiz  3 

177 

8.339 

1.292 

-.799 

-596 

4.0 

10.0 

Progress  Check  4  Quiz  1 

178 

8.062 

.928 

-  .983 

.858 

5.0 

9.0 

Progress  Check  4  Quiz  2 

178 

7.916 

1.089 

-1.054 

1.152 

4.0 

9.0 

Progress  Check  4  Quiz  3 

177 

2.960 

1.222 

-  .850 

-346 

0.0 

4.0 

Progress  Check  5  Quiz  1 

180 

2.833 

.388 

-2.089 

3.276 

1.0 

3.0 

Progress  Check  5  Quiz  2 

180 

3.806 

.474 

-2.773 

8.979 

1.0 

4.0 

Progress  Check  S  Quiz  3 

180 

3.228 

.944 

-1.034 

.226 

0.0 

4.0 

Progress  Check  5  Quiz  4 

180 

9.350 

1.284 

-2.314 

5.406 

3.0 

10.0 

Progress  Check  6  Quiz  1 

177 

11.412 

1.727 

-  .429 

-.275 

7.0 

14.0 

Progress  Check  6  Quiz  2 

170 

5.153 

.967 

-1.067 

.552 

2.0 

6.0 

Progress  Check  7  Quiz  1 

181 

5.182 

.934 

-1.158 

1.133 

2.0 

6.0 

Progress  Check  7  Quiz  2 

181 

9.442 

.812 

-1.858 

4.999 

5.0 

10.0 

Progress  Check  7  Quiz  3 

180 

2.983 

.942 

-  .655 

-  .265 

0.0 

4.0 

Progress  Check  7  Quiz  4 

180 

3.422 

.724 

-.929 

-  .167 

1.0 

4.0 

Progress  Check  8  Quiz  1 

179 

8.626 

1.267 

-1,056 

1.255 

4.0 

10.0 

Progress  Check  8  Quiz  2 

179 

3.095 

.916 

-  .811 

.055 

0.0 

4.0 

Progress  Check  8  Quiz  3 

179 

5.883 

1.454 

-.381 

-.653 

2.0 

8.0 

Progress  Check  9  Quiz  1 

175 

3^83 

.580 

-1.222 

1.472 

1.0 

4.0 

Progress  Check  9  Quiz  2 

175 

7.337 

1.206 

-  .496 

-  ,219 

4.0 

9.0 

Progress  Check  9  Quiz  3 

175 

4J37 

.869 

-1.814 

2.408 

1.0 

5.0 

“The  Progress  Checks  are  from  Curriculum  2. 
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Tabic  121 

CTimrr’jra titles  and  Tacior  l.oaGsng^  tor  2'*2"0  Vanabies  ■  Former  Curriculum 


Variable® 

Communality 

1 

FSG 

Factor  Loadings 

2  3  4  5 

Block  3A  Block  5B  Block  3B  Block  5A 

Block  1  (Nat.  Air  Traffic  Training  Prog.)  Hours 

.237 

.039 

-.271 

-.253 

.110 

-.293 

Block  2  f  Control  Tower  Proc.)  Hours 

.129 

-.286 

-.049 

-.055 

-.028 

-.202 

Block  3A  (Basic  Tower  Op.)  Hours 

.322 

-.041 

-315 

-.029 

.103 

-.209 

Block  3B  (Adv^inced  Tower  Op.)  Hours 

.658 

.019 

.064 

.119 

.799 

-.025 

Block  4  (Radar  Approach  Control  Proc.)  Hours 

.471 

-.627 

.153 

.018 

.005 

-.232 

Block  5A  (Basic  Approach  Control  Op.)  Hours 

.149 

-.090 

-.012 

-.091 

-.033 

-.363 

Block  5B  (Advanced  Approach  Control  Op.)  Hours 

.212 

.028 

-.039 

-.455 

-.011 

-.045 

Block  1  (Nat.  Air  Traffic  Training  Prog.)  Written  Test 

.444 

.388 

.272 

.323 

.199 

.276 

Block  2  (Control  Tower  Proc.)  Written  Test 

.417 

.622 

.087 

.030 

-.101 

.106 

Block  4  (Radar  Approach  Control  Proc.)  Written  Test 

.685 

.794 

.101 

.146 

.023 

.150 

Block  3A  (Basic  Tower  Op.)  Perf.  Test 

.297 

-.094 

318 

-.079 

-.111 

.038 

Block  3B  (Adv-anced  Tower  Op.)  Perf.  Test 

.646 

.013 

-.092 

.040 

-.795 

-.055 

Block  5A  (Basic  Approach  Control  Op.)  Perf.  Test 

.209 

.035 

-.064 

-.003 

.020 

.451 

Block  5B  (Advanced  Approach  Control  Op.)  Perf.  Test 

SA\ 

.005 

-.031 

.734 

.033 

-.015 

Proeress  Cheek  1  Quiz  1 

.274 

.253 

.230 

.244 

-.246 

.192 

P:  rress  Cheek  2  Quiz  1 

.2S4 

.143 

.455 

-.010 

.080 

-.001 

Progress  Check  2  Quiz  2 

.307 

.093 

.487 

.196 

.120 

-.091 

Progress  Check  2  Quiz  3 

.248 

.112 

.463 

.137 

.049 

-.018 

Progress  Check  3  Quiz  1 

.176 

.110 

.070 

.392 

-.068 

.024 

Progress  Check  3  Quiz  2 

.348 

.217 

328 

.020 

.086 

-.124 

Progress  Check  3  Quiz  3 

.092 

.271 

.076 

.002 

-.002 

-.113 

Progress  Check  4  Quiz  1 

.271 

.057 

.360 

.319 

-.188 

-.038 

Progress  Cheek  4  Quiz  2 

.221 

.162 

.333 

-.028 

.066 

.279 

Progress  Cheek  4  Quiz  3 

.171 

.256 

.121 

.179 

.067 

.233 

Progress  Cheek  5  Quiz  1 

.149 

.055 

.028 

.231 

.072 

-.295 

Progress  Check  5  Quiz  2 

.189 

.316 

.024 

.133 

.027 

-.266 

Progress  Cheek  5  Quiz  3 

.256 

.163 

.432 

.190 

.040 

-.072 

Progress  Check  5  Quiz  4 

.103 

.146 

.054 

.274 

.008 

.062 

Progress  Cheek  6  Quiz  1 

.331 

.241 

.304 

.414 

.053 

-.083 

Progress  Check  6  Quiz  2 

.210 

.256 

.290 

.178 

.088 

.145 

Progress  Check  7  Quiz  1 

.114 

.297 

.148 

-.027 

.048 

-.032 

Progress  Check  7  Quiz  2 

.098 

-.061 

.182 

.237 

.045 

-.050 

Progress  Cheek  7  Quiz  3 

.231 

.385 

.175 

.206 

-.088 

-.039 

Progress  Check  7  Quiz  4 

.242 

.454 

.098 

-.087 

-.017 

-.136 

Progress  Cheek  8  Quiz  1 

.262 

365 

315 

.150 

.086 

.020 

Progress  Cheek  8  Quiz  2 

.205 

.287 

361 

.173 

-.015 

.157 

Progress  Check  8  Quiz  3 

.254 

.178 

367 

.279 

.018 

.097 

Progress  Cheek  9  Quiz  1 

.137 

-.090 

.203 

.015 

-.135 

,264 

Progress  Check  9  Quiz  2 

.190 

.144 

338 

.069 

-.(XT? 

.224 

Progress  Check  9  Quiz  3 

.082 

-.047 

-.001 

.203 

.080 

.  .180 

Note,  n  =  141. 

*The  Progress  Checks  arc  from  Curriculum  2. 
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Table  12; 

Factor  Score  Coefficients  for  2~2'0  \'ariabic&  -  Former  Curriculum 


Factor  Score  Coefficients 
1  2  3  4  5 

Variable^  FSG  Block  3A  Block  5B  Block  3B  Block  5A 


Block  1  (Nat.  Air  Traffic  Training  Prog.)  Hours 

.063 

-.057 

-.069 

.033 

-.124 

Block  2  (Control  Tower  Proc.)  Hours 

-.014 

.004 

.029 

-.044 

-.066 

Block  3A  (Basic  Tower  Op.)  Hours 

.058 

-.180 

.039 

.052 

-.094 

Block  3B  (Advanced  Tower  Op.)  Hours 

-.032 

-.031 

.074 

.474 

-.038 

Block  4  (Radar  Approach  Control  Proc.)  Hours 

-.119 

.143 

.054 

.002 

-.120 

Block  5A  (Basic  Approach  Control  Op.)  Hours 

.013 

.022 

-.030 

-.020 

-.163 

Block  5B  (Advanced  Approach  Control  Op.)  Hours 

.042 

.016 

-.116 

-.023 

-.004 

Block  1  (Nat.  Air  Traffic  Training  Prog.)  Written  Test 

.055 

.031 

.099 

.020 

.159 

Block  2  (Control  Tower  Proc.)  Written  Test 

.234 

-.034 

-.027 

-.083 

.005 

Block  4  (Radar  Approach  Control  Proc.)  Written  Test 

.472 

-.078 

.009 

.022 

.024 

Block  3A  (Basic  Tower  Op.)  Perf.  Test 

-.047 

.176 

-.061 

-.065 

.012 

Block  3B  (Advanced  Tower  Op.)  Perf.  Test 

.006 

-.054 

.099 

-.439 

-.064 

Block  5A  (Basic  Approach  Control  Op.)  Perf.  Test 

.019 

-.044 

.012 

.015 

.232 

Block  5B  (Advanced  Approach  Control  Op.)  Perf.  Test 

-.048 

-.136 

.493 

-.021 

-.034 

Progress  Check  1  Quiz  1 

.009 

.CM5 

.034 

-.069 

.080 

Progress  Check  2  Quiz  1 

-.013 

.126 

-.070 

.029 

-.018 

Progress  Check  2  Quiz  2 

-.022 

.147 

.021 

.039 

-.077 

Progress  Check  2  Quiz  3 

-.008 

.120 

.016 

.012 

-.039 

Progress  Check  3  Quiz  1 

-.016 

-.014 

.121 

-.023 

-.001 

Progress  Check  3  Quiz  2 

.028 

.180 

-.056 

.021 

-.114 

Progress  Check  3  Quiz  3 

.019 

.003 

-.019 

.031 

-.076 

Progress  Check  4  Quiz  1 

-.035 

.083 

.086 

-.064 

-.033 

Progress  Check  4  Quiz  2 

-.010 

.082 

-.046 

.017 

.124 

Progress  Check  4  Quiz  3 

.002 

-.004 

.030 

.016 

.089 

Progress  Check  5  Quiz  1 

.013 

.003 

.078 

-.002 

-.125 

Progress  Check  5  Quiz  2 

.066 

-.022 

.055 

.013 

-.150 

Progress  Check  5  Quiz  3 

.032 

.115 

.028 

-.012 

-.077 

Progress  Check  5  Quiz  4 

-.010 

-.008 

.038 

.012 

.016 

Progress  Check  6  Quiz  1 

.023 

.051 

.123 

-.001 

-.098 

Progress  Check  6  Quiz  2 

.020 

.056 

,040 

-.009 

.043 

Progress  Check  7  Quiz  1 

.085 

.017 

-.007 

.001 

-.021 

Progress  Check  7  Quiz  2 

-.035 

.024 

.066 

.010 

-.022 

Progress  Check  7  Quiz  3 

.096 

.021 

.009 

-.011 

-.057 

Progress  Check  7  Quiz  4 

.112 

.023 

-.061 

-.006 

-.116 

Progress  Check  8  Quiz  1 

.064 

.066 

-.002 

.001 

-.015 

Progress  Check  8  Quiz  2 

.043 

.032 

.022 

.018 

.041 

Progress  Check  8  Quiz  3 

.006 

.079 

.050 

.003 

.029 

Progress  C^eck  9  Quiz  1 

-.064 

.054 

.012 

-.010 

.108 

Progress  Check  9  Quiz  2 

-.018 

.081 

-.027 

-.014 

.092 

Progress  Check  9  Quiz  3 

-.002 

00 

o 

.065 

-.007 

.  .085 

*The  Progress  Checks  are  from  Curriculum  2. 


We  computed  several  composite  variables,  including  a  measure  for  each  of 
Factors  2  through  5.  We  defined  the  composites  as  follows:  61ock3A 
(standardized  Block  3A  test  score  -  standardized  Block  3A  hours)  for  Factor  2, 
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BlockSB  (standardized  Block  5B  test  score  -  standardized  Block  5B  hours)  for 
Factor  3.  BlockSB  (standardized  Block  3B  test  score  -  standardized  Block  3B 
hours)  for  Factor  4,  and  BlockSA  (standardized  Block  5A  test  score  -  standardized 
Block  5A  hours)  for  Factor  5.  We  also  computed  the  sum  of  the  hours  for  the  4 
performance  tests,  the  sum  of  the  hours  for  the  3  written  tests,  the  mean  of  the 
4  performance  test  scores,  and  the  mean  of  the  3  written  test  scores  (Final 
School  Grade  computed  from  the  initial  test  scores  or  FSG2).  We  also 
standardized  each  of  the  above  4  composites  and  added  them  to  obtain  a  combined 
hours  variable  (the  sum  of  the  standardized  hours  for  the  performance  tests  and 
the  standardized  hours  for  the  written  tests)  and  a  combined  test  variable  (the 

sum  of  the  standardized  performance  test  sum  and  the  standardized  Final  School 
Grade) . 

Table  123  shows  descriptive  statistics  on  these  composites. 


Table  123 

_Descnptive  Statistics  for  27230  Composite  Variables  -  Former  Curriculum 


Composite 

Reliability® 

Mean 

Std  Dev 

Skewness  Kurtosis 

Minimum  Maximum 

FSG 

FSG2“ 

BlockSA' 

BlockSB'^ 

BlockSB® 

BlockSA^ 

Perf.  Hours* 

Written  Hours^ 

Perf.  Tests^ 

N/A 

.73 

-5S 

^7 

.79 

.40 

.42 

.39 

.32 

82.486 

82.044 

.000 

.000 

.000 

.000 

324331 

290.204 

.928 

S.889 

6369 

1.781 

1,625 

1.869 

1.702 

31.318 

28.971 

.118 

.019 

-.422 

-3.046 

-2.341 

-8.392 

-2.497 

1.295 

1.635 

-1.197 

-.646 

.255 

9,157 

5.373 

82,263 

5.234 

1.904 

4302 

.073 

70.000 
58.000 
-8,650 
-7.030 
-18.280 
-6.260 
240.000 
204.000 
300  ■ 

97.000 

97.330 

1.660 

2350 

3,740 

1.420 

444.000 

425.000 

1.000 

Note.  N  =  222  - - - - - 

These  arc  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
°FSG2:  Mean  of  initial  test  scores  for  Blocks  1,  2,  and  4. 

^BlockSA:  Standardized  Block  3A  test  score  -  standardized  Block  3A  hours 
BlockSB:  Standardized  Block  SB  test  score  -  standardized  Block  SB  hours 
•BlockSB:  Sundardizcd  Block  3B  test  score  -  standardized  Block  3B  hours 
BlockSA:  Standardized  Block  SA  test  score  -  standardized  Block  SA  hours 
*Perf.  Hours:  Sum  of  hours  for  Blocks  3A,  3B,  SA  and  SB. 

^Writ.  Hours;  Sum  of  hours  for  Blocks  1.  2,  and  4, 

^Perf.  Tests:  Mean  of  (initial)  test  scores  for  Blocks  3A,  3B,  SA,  and  SB. 
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We  also  computed  Pearson  correlation  coefficients  for  the  composite 
variables.  Table  124  displays  correlations  among  the  composites.  FSG  and  FSG2 
correlate  .97,  which  offers  no  support  for  using  initial  test  scores  in  computing 
FSG. 


Table  124 

Correlations  Among  27230  Composites  and  Factors  •  Former  Curriculum 


Composite^ 

Fad 

Fac2 

Fac3 

Fac4 

Fac5 

FSG 

FSG2 

Factor  2 

.07 

Factor  3 

.04 

.10 

Factor  4 

-.00 

.02 

.01 

Factor  5 

.06 

.05 

-.01 

-.01 

FS 

.83 

.26 

.27 

.CM 

.22 

.89 

.21 

.21 

.03 

.26 

.97 

BIock3A' 

-.03 

.67 

-.03 

-.13 

.19 

.11 

.12 

Test3A 

-.10 

59 

-.09 

-.12 

.05 

.02 

.02 

BlockSB'^ 

-.W 

.02 

.83 

.03 

-.02 

.14 

.09 

TcstSB 

-.02 

-.02 

.86 

.03 

-.06 

.16 

.11 

BIock3B* 

-.01 

-.10 

-.06 

-.97 

-.02 

-.09 

-.08 

Tcst3B 

.02 

-.11 

.05 

-.89 

-.07 

-.06 

-.05 

BIockSA^ 

.07 

-.03 

.04 

.03 

.62 

.10 

.10 

TestSA 

.01 

-.06 

-.CM 

.02 

56 

.CM 

.03 

Perf.  Hours^ 

-.03 

-.37 

-.34 

.16 

-.35 

-.18 

-.17 

Writ.  Hours^ 

-J9 

-.05 

-.12 

.06 

-.47 

-53 

-.64 

Perf.  Tcsts^ 

-.05 

.23 

53 

-.19 

.36 

.14 

.10 

Perf. 

Writ. 

Composite® 

Block3A 

Test3A 

BlockSB 

TestSB 

BlcxkSB 

Tcst3B 

BlockSA 

TcstSA 

Hours 

Hours 

Tcsi3A 

.88 

BlockSB^ 

.06 

-.05 

TestSB 

-.01 

-.09 

.86 

Block3B® 

-.00 

-.01 

-.06 

-.08 

Test3B 

-.03 

-.02 

-.01 

-.03 

.90 

BlockSA^ 

-.02 

-.10 

-.02 

-.07 

-.00 

-.03 

TestSA 

-.03 

-.09 

-.07 

-.12 

.00 

-.03 

.86 

Perf.  Hours^ 

-58 

-.30 

-56 

-.28 

-.12 

-.07 

-.35 

-.17 

Writ,  Houis^ 

-.15 

-.05 

-.00 

-.05 

-.01 

.02 

-.13 

-.05 

.16 

Perf.  Tests^ 

.43 

.40 

52 

56 

.11 

.14 

,49 

56 

-.48 

-.09 

Note.  N  *  140. 

^These  are  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^FSG2:  Mean  of  initial  test  scores  for  Blocks  1,  2,  and  4. 

^Block3A:  Standardized  Block  3A  test  score  -  standardized  Block  3A  hours 
^BIockSB:  Standardized  Block  SB  test  score  -  standardized  Block  SB  hours 
^BIock3B:  Standardized  Block  3B  test  score  -  sundardized  Block  3B  hours 
^BlockSA:  Standardized  Block  SA  test  score  •  standardized  Block  SA  hours 
^Perf.  Hours:  Sum  of  hours  for  Blocks  3A.  3B,  SA,  and  SB. 

^Writ.  Hours:  Sum  of  hours  for  Blocks  1,  2,  and  4. 

^Perf.  Tests:  Mean  of  (initial)  test  scores  for  Blocks  3A,  3B,  SA»  and  SB. 


193 


FSG2  correlates  highest  (.89)  with  Factorl,  followed  closely  by  FSG  (.83). 
Block3A  correlates  highest  (.67)  with  Factor2,  followed  by  Test3A  (.59).  TestSB 
correlates  highest  (.86)  with  Factors,  followed  closely  by  BlockBB  (.83). 
BlockSB  correlates  highest  (-.97)  with  Factor4,  followed  by  Test3B  (-.89). 
Similarly,  BlockBA  correlates  highest  (.62)  with  Factors,  followed  by  TestSA 
(.56). 

The  high  correlations  of  BlockBB  and  TestBB  with  Factors,  and  the  high 
correlations  of  BlockSB  and  TestSB  with  Factor4  indicate  that  these  may  indeed 
represent  real  independent  dimensions  of  performance.  The  small  differences  in 
correlation  between  the  full  block  measure  (test  score  -  hours)  and  the  test 
score  by  itself  suggest  that  the  TestBB  score  and  the  TestSB  score  by  themselves 
may  represent  Factor  3  and  4  adequately.  Further,  these  two  performance  tests 
represent  advanced  aspects  of  the  27230  course:  TestSB  Advanced  Tower  Operation 
and  TestBB  Advanced  Approach  Control  Operation.  Moreover,  TestSB  and  TestBB  are 
independent  of  one  another  (r  =  -.03).  The  mean  of  TestSB  and  TestBB  correlates 
.85  with  Factors  and  -.20  with  Factor4,  indicating  that  researchers  should  not 
combine  these  two  tests. 

The  low  correlations  between  BlockSA  and  TestSA  with  Factor2  and  between 
BlockBA  and  TestBA  with  Factor  5  indicate  that  these  composites  do  not  represent 
the  factors  well.  None  of  the  other  composites  (performance  hours,  written  test 
hours,  performance  test  mean)  represented  any  factors  well. 

These  correlational  results  suggest  that  researchers  should  use  FSG, 
TestSB,  and  TestBB  to  describe  27230  school  performance  in  the  former  curriculum. 

Current  Curriculum.  Table  125  shows  descriptive  statistics  on  variables 
in  the  current  curriculum.  In  the  current  curriculum,  scores  for  all  but  33 
students  were  missing  on  the  Block  3B  (Advanced  Tower  Operation)  Performance  Test 
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due  to  failure  of  a  simulator,  thus,  we  did  not  further  analyze  this  test  or  its 
corresponding  hours.  We  also  did  not  analyze  Progress  Check  3  Quiz  la 
(Curriculum  3)  (IFR  separation),  due  to  it  having  only  189  cases;  Progress  Check 
3  Quiz  la  (Curriculum  4)  (IFR  separation)  (only  23  cases);  and  Progress  Check  3 
Quiz  If  (Curriculum  4  only)  (Airport  Surveillance  Radar)  (only  24  cases). 

We  initially  conducted  factor  analysis  of  all  variables,  including  quizzes, 
in  the  current  curriculum.  While  the  scree  test  indicated  one  strong  factor, 
none  of  the  factor  solutions  provided  clearly  interpretable  results.  The  only 
solution  with  a  clear  pattern  of  factor  loadings  involved  analysis  of  just  the 
hours,  written  tests,  and  performance  tests,  excluding  the  quizzes.  Table  126 
shows  commonalities  and  factor  loadings  for  the  five-factor  solution,  while  Table 
127  shows  the  factor  score  coefficients.  The  first  factor  involves  the  sum  of 
the  written  tests  except  Block  2,  minus  Block  4  hours.  The  remainder  of  the 
factors  are  doublets  consisting  of  the  performance  test  scores  minus  the 
corresponding  hours,  and  Block  2  test  score  minus  Block  2  hours.  This  solution 
resembles  that  for  the  former  curriculum,  except  for  the  written  Test  2  standing 
as  a  factor  and  the  lack  of  a  factor  for  Block  3B.  Accordingly,  we  developed 
composite  variables  to  see  if  they  would  represent  the  obtained'  factors. 
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Tabic  1:5 

Dcscnrttve  Statistics  for  2^230  N'anahles  »  Current  Cumculum 


V'anablc^  _n  Mean  Std  Dev  Skewness  Kunosis  Minimum  Maximum 


Block  1  (ATC  Fundamentals)  Hours 

416 

88.493 

19.606 

2.382 

6.289 

54.800 

200.000 

Block  2  ('Control  Tower  Proc.)  Hours 

399 

73.797 

23.636 

1.014 

.164 

20300 

152.000 

Block  3A  (Basic  Tower  Op.)  Hours 

355 

81.899 

21.767 

.334 

.653 

11000 

153300 

Block  3B  (Advanced  Tower  Op.)  Hours 

33 

45.848 

15.093 

2.036 

2.718 

29300 

89300 

Block  4  (Radar  Approach  Coniroi  Proc.)  Hours 

340 

76.222 

16.753 

3.687 

13.419 

63300 

175300 

Block  5A  (Basic  Approach  Control  Op.)  Hours 

326 

50.458 

11.397 

2.672 

6.061 

84.000 

96.000 

Block  5B  (Advanced  Approach  Control  Op.)  Hours 

324 

145.277 

19399 

.344 

5.424 

42.600 

237.000 

Block  6  (Fed.  Control  Tower  Oper.  Criteria)  Hours 

321 

31.126 

6418 

.309 

.275 

18.000 

53.000 

Block  1  (ATC  Fundamentals)  Written  Test 

410 

80.954 

8.903 

-.607 

.944 

40.000 

100.000 

Block  2  (Control  Tower  Proc.)  Written  Test 

392 

75.648 

12.079 

-313 

-.203 

40.000 

100.000 

Block  4  (Radar  Approach  Control  Proc.)  Written  Test 

341 

82.273 

9.849 

-.911 

1.082 

42,000 

98.000 

Block  6  (Fed.  Control  Tower  Oper.  Critena)  Written  Test 

315 

86.733 

5.450 

-.424 

-.176 

70.000 

99.000 

Block  3A  (Basic  Tower  Op.)  Perf.  Test 

342 

.965 

.184 

-5.076 

23.902 

.000 

1.000 

Block  3B  (Advanced  Tower  Op.)  Perf.  Test 

33 

.879 

.331 

4.170 

-2.433 

.000 

1.000 

Block  5A  (Basic  Approach  Control  Op.)  Perf.  Test 

327 

.893 

.310 

-2354 

4.550 

.000 

1.000 

Block  SB  (Advanced  Approach  Control  Op.)  Perf.  Test 

322 

.885 

.319 

-2.426 

3.912 

.000 

1.000 

PC  1  Quiz  la  (Curr.  3)  (IFR  separation) 

189 

7.910 

1.030 

00 

318 

4.000 

9.000 

1  1  Quiz  la  (Curr.  4)  (IFR  separation) 

23 

7.957 

1.107 

-1.229 

1.193 

5.000 

9.000 

1  ^  1  Quiz  lb  (IFR  depanurcs) 

212 

5.024 

1.150 

-1.199 

.948 

1.000 

6.000 

PC  2  Quiz  Ic  (holding  instructions) 

223 

5.614 

.640 

-1.739 

3.029 

3.000 

6.000 

PC  2  Quiz  Id  (IFR  arrivals) 

223 

8.103 

1.050 

-.986 

.111 

5.000 

9.000 

PC  3  Quiz  le  (general  radar  procedures) 

235 

4328 

.706 

-1.459 

1.702 

2.000 

5.000 

PC  3  Quiz  If  (Curriculum  4  only)  (ASR) 

24 

3383 

384 

-1.067 

.295 

2.000 

4.000 

PC  3  Quiz  If  (Curr  3),  Ig  (Curr  4)  (electronic  combat) 

235 

4.277 

1.115 

-1.272 

307 

.000 

5.000 

PC  3  Quiz  Ig  (Curr  3),  Ih  (Curr  4)  (beacon  codes) 

235 

8.817 

1.119 

-.684 

-.438 

6.000 

10.000 

PC  4  Quiz  Ih  (Curr  3),  li  (Curr  4)  (radar  identification) 

242 

7.888 

1.145 

-1.184 

1.203 

4.000 

9.000 

PC  4  Quiz  li  (Curr  3)»  Ij  (Curr  4)  (vector  aircraft) 

242 

7.921 

1.130 

-.959 

.416 

4.000 

9.000 

PC  4  Quiz  Ij  (Curr  3),  Ik  (Curr  4)  (radar  separation) 

242 

8.992 

1.637 

-.856 

.453 

3.000 

11.000 

PC  5  Quiz  Ik  (Curr  3).  11  (Curr  4)  (low  altitude  alerts) 

243 

3.774 

.457 

-1.827 

2.489 

2.000 

4.000 

PC  5  Quiz  11  (Curr  3),  Im  (Curr  4)  (additional  services) 

243 

7.160 

.878 

-.983 

1.250 

3.000 

8.000 

PC  5  Quiz  Im  (Curr  3).  In  (Curr  4)  (terminal  radar  prog.) 

243 

8.630 

1.261 

-.870 

393 

4.000 

10.000 

PC  6  Quiz  In  (Curr  3),  lo  (Curr  4)  (radar  arrivals) 

242 

12.930 

1.369 

-1.369 

1381 

7.000 

14.000 

PC  6  Quiz  lo  (Curr  3),  Ip  (Curr  4)  (aircraft  altitudes) 

242 

3.434 

.750 

-1.143 

360 

1.000 

4.000 

PC  6  Quiz  Ip  (Curr  3),  Iq  (Curr  4)  (transfer  radar  ID) 

242 

10.223 

1.771 

-1.224 

1375 

3.000 

12.000 

PC  7  Quiz  Iq  (Curr  3),  Ir  (Curr  4)  (radar  arrivals) 

245 

13.865 

1.139 

-.958 

.901 

9.000 

15.000 

PC  7  Quiz  Ir  (Curr  3),  Is  (Curr  4)  (visual  approaches) 

245 

3.102 

.920 

-.905 

.352 

.000 

4.000 

PC  7  Quiz  Is  (Curr  3),  It  (Curr  4)  (contact  approaches) 

245 

3322 

.644 

-1.199 

1.119 

1.000 

4.000 

PC  8  Quiz  It  (Curr  3),  lu  (Curr  4)  (radar  approaches) 

245 

4.016 

.864 

-.725 

.205 

1.000 

5.000 

PC  8  Quiz  lu  (Curr  3),  Iv  (Curr  4)  (ASR  approaches) 

245 

6.318 

1.247 

-.367 

-.606 

3.000 

8.000 

PC  9  Quiz  Iv  (Curr  3),  Iw  (Curr  4)  (PAR  indicators) 

246 

3369 

.600 

-1.067 

.128 

2.000 

4.000 

PC  9  Quiz  Iw  (Curr  3),  lx  (Curr  4)  (PAR  approaches) 

246 

6.439 

1.165 

-.429 

-.368 

3.000 

8.000 

•PC:  Progress  Check 
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Tabic  126 

Communanties  and  Factor  Loadintn;  for  2**230  Variables  -  Current  Curriculum 


Variable 

Communaiitv' 

1 

FSG 

Factor  Loadings 

2  3  4  5 

Block  5A  Block  2  Block  5B  Block  3A 

Block  1  (ATC  Fundamentals)  Hours 

.106 

..126 

.037 

-.013 

.031 

-.296 

Block  2  (Control  Tower  Proc.)  Hours 

.435 

..154 

.036 

-.636 

-.028 

-.067 

Block  3A  (Basic  Tower  Op.)  Hours 

.271 

..020 

.201 

-.105 

-.166 

-.438 

Block  4  (Radar  Approach  Control  Proc.)  Hours 

.2(A 

-.500 

-.062 

-.064 

.041 

.067 

Block  5A  (Basic  Approach  Control  Op.)  Hours 

.737 

.021 

.856 

-.026 

-.007 

-.059 

Block  5B  (Advanced  Approach  Control  Op.)  Hours 

.307 

.,027 

.079 

-.145 

-322 

-.076 

Block  6  (Fed.  Control  Tow'cr  Oper.  Criteria)  Hours 

.033 

.013 

.039 

-.108 

-.108 

-.092 

Block  1  (ATC  Fundamentals)  Written  Test 

.317 

.376 

-.038 

.205 

.124 

.341 

Block  2  (Control  Tower  Proc.)  Written  Test 

.630 

.216 

-.012 

.749 

.148 

.008 

Block  4  (Radar  Approach  Control  Proc.)  Written  Test 

.761 

.840 

-.026 

.163 

.072 

.150 

Block  6  (Fed.  Control  Totver  Oper.  Criteria)  Written  Test 

.324 

.418 

-.087 

.163 

.164 

.298 

Block  3A  (Basic  Tower  Op.)  Perf.  Test 

.245 

..032 

.037 

-.015 

.101 

.482 

Block  5A  (Basic  Approach  Control  Op.)  Perf.  Test 

.593 

.012 

..751 

.048 

.132 

.094 

Block  SB  (Advanced  Approach  Control  Op.)  Perf.  Test 

.613 

.086 

-.021 

-.008 

.771 

.099 

Note.  N  =  311. 

Table  127 

Factor  Score  Coefficients  for  27230  Variables  -  Current  Curriculum 

Factor 

Score  Coefficients 

1 

2 

3 

4 

5 

Variable 

FSG 

Block  5A  Block  2 

Block  5B  Block  3A 

Block  1  (ATC  Fundamentals)  Hours 

-.036 

-.008 

.006 

.039 

-.164 

Block  2  (Control  Tower  Proc.)  Hours 

.045 

-.018 

-.344 

.049 

-.019 

Block  3A  (Basic  Tower  Op.)  Hours 

.066 

.014 

-.011 

-.034 

-.291 

Block  4  (FUdar  Approach  Control  Proc.)  Hours 

-.137 

-.004 

.031 

.036 

.090 

Block  5A  (Basic  Approach  Control  Op.)  Hours 

.017 

.630 

-.000 

.058 

.039 

Block  5B  (Advanced  Approach  Control  Op.)  Hours 

.046 

-.003 

-.032 

-.236 

.011 

Block  6  (Fed.  Control  Tower  Oper.  Criteria)  Hours 

.030 

-.003 

-.030 

-.015 

-.044 

Block  1  (ATC  Fundamentals)  Written  Test 

.049 

.000 

.045 

-.005 

.199 

Block  2  (Control  Tower  Proc.)  Written  Test 

-.035 

.012 

.602 

.056 

-.113 

Block  4  (Radar  Approach  (Control  Proc.)  Written  Test 

.754 

-.010 

-.102 

-.033 

-.003 

Block  6  (Fed.  Control  Tower  Oper.  Criteria)  Written  Test 

.108 

.004 

-.005 

.012 

.157 

Block  3A  (Basic  Tower  Op.)  Perf.  Test 

-.086 

.041 

-.016 

-.010 

.343 

Block  5A  (Basic  Approach  Control  Op.)  Perf.  Test 

-.006 

-.348 

-.013 

.038 

.007 

Block  5B  (Advanced  Approach  Control  Op.)  Perf.  Test 

.008 

.034 

-.100 

.662 

-.045 

We  computed  the  following  composite  variables:  FSG2,  BlockSA  (standardized 
TestSA  -  standardized  BlockSA  hours),  BlockZ  (standardized  Test2  -  standardized 
Block2  hours),  BlockSB  (standardized  TestSB  -  standardized  BlockSB  hours),  and 
BlockSA  (standardized  TestSA  -  standardized  BlockSA  hours).  We  also  computed  the 
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sum  of  the  hours  for  the  3  performance  tests  (3A,  5A,  5B),  the  sum  of  the  hours 
for  the  4  written  tests  (1,  2,  4,  and  6),  the  mean  of  the  3  performance  test 
scores,  and  the  mean  of  the  4  written  test  scores  (Final  School  Grade,  computed 
from  the  initial  test  scores,  FSG2).  We  also  standardized  each  of  the  above 
composites  and  added  them  to  obtain  a  combined  hours  variable  (the  sum  of  the 
standardized  hours  for  the  performance  tests  and  the  standardized  hours  for  the 
written  tests)  and  a  combined  test  variable  (the  sum  of  the  standardized 
performance  test  sum  and  the  standardized  Final  School  Grade).  Table  128  shows 
descriptive  statistics  for  these  composites. 

Table  129  displays  correlations  among  the  composites.  Since  FSG  and  FSG2 
correlate  .97,  this  offers  no  support  for  using  initial  test  scores  to  compute 
FSG.  FSG  and  FSG2  each  correlate  .74  with  Factorl,  a  correlation  no  doubt 
attenuated  by  the  low  loading  of  Test2  on  the  factor.  BlockSA  correlates  highest 


Table  128 

Descriptive  Statistics  for  27230  Composite  Variables  -  Current  Curriculum 


Composite 

Reliability® 

Mean 

Std  Dev 

Skewness 

Kurtosis 

Minimum  Maximum 

FSG 

N/A 

83.642 

5.115 

.336 

-.460 

72.000 

98.000 

.76 

82.986 

5.658 

.060 

-.239 

66.750 

97-500 

BlockSA' 

.80 

.000 

1.814 

-3.054 

8.493 

-7.840 

1.860 

BlodcZ** 

.69 

.000 

1.736 

-1.110 

J63 

-5.990 

2.880 

BlockSB® 

.62 

.000 

1.692 

-1.935 

5.277 

-7.960 

5.800 

BlockSA^ 

.42 

.000 

1.608 

-3.321 

15.458 

-9.160 

2.240 

Perf.  Hours® 

-58 

276.222 

34.316 

.924 

2.124 

159.600 

404.500 

Written  Hours^ 

.35 

256,453 

28.120 

1.628 

2.820 

209.100 

380300 

Perf.  Tests^ 

.57 

.932 

.155 

-2.219 

4.277 

.330  ‘ 

1.000 

Note.  &  »  307 


“These  arc  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^FSG2:  Mean  of  initial  test  scores  for  Blocks  1,  2,  4,  and  6. 

®BIock5A:  Standardized  Block  5A  Performance  Test  -  standardized  Block  5A  hours 
‘*Block2:  Standardized  Block  2  Written  Test  -  standardized  Block  2  hours 
•BlockSB:  Standardized  Block  5B  Performance  Test  -  standardized  Block  5B  hours 
%lock3A:  Standardized  Block  3A  Performance  Test  -  standardized  Block  3A  hours 
*Pcrf.  Hours:  Sum  of  hours  for  Blocks  3A.  SA,  and  5B. 

^Writ.  Hours:  Sum  of  hours  for  Blocks  1,  2,  4»  and  6 

^Perf.  Tests:  Mean  of  initial  test  scores  for  Blocks  3A  5 A,  and  5B. 
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Table 

Corrciationg  Among  2"2r^0  Composites  and  Factors  -  Current  Curriculum 


Composite- 

Fac  1 

Fac  2 

Fac  3 

Fac  4 

Fac  5 

FSG 

FSG2 

Factor  2 

.01 

Factor  3 

.11 

-.01 

Factor  4 

.01 

-00 

.02 

Factor  5 

.15 

-.06 

.02 

.11 

FSG 

.74 

-.03 

i3 

.15 

.39 

FSG2*=’ 

.74 

-.04 

.63 

.17 

.35 

.97 

Blocks  A 

-.00 

-.99 

.03 

.07 

.10 

.07 

.07 

TestSA 

.02 

-.84 

.05 

.15 

.12 

.11 

.11 

Block2“ 

.23 

-.02 

.97 

.08 

.05 

.57 

.67 

Test2 

.24 

.01 

.92 

.14 

-.02 

.62 

.71 

BlockSB* 

.06 

-.05 

.07 

.95 

.12 

.19 

.21 

TestSB 

.07 

.00 

-.06 

.95 

.11 

.13 

.15 

Block3A^ 

-.01 

-.08 

.05 

.18 

.85 

.19 

.17 

Test3A 

-.04 

.11 

-.06 

.08 

.71 

.08 

.06 

Perf.  Hours6 

-.02 

.47 

-.18 

-.48 

-.47 

-.25 

-.25 

W'riiten  Hours" 

-.41 

.02 

-.58 

.03 

-.21 

-.43 

-iO 

Perf.  Tests- 

.04 

-.48 

-.03 

.72 

.37 

.17 

.18 

Perf. 

Writ. 

Cc  -.posiie® 

BlockSA 

TestSA 

Block2 

Tesi2 

BlockSB 

TestSB 

B]ock3A 

Test3A  Hrs 

Hrs 

TestSA 

.91 

Blocia"^ 

.05 

.07 

Test2 

.02 

.06 

.87 

BlockSB® 

.10 

.14 

.13 

.15 

TestSB 

.06 

.14 

.03 

.05 

.85 

Bloclc3A^ 

.11 

.11 

.07 

.07 

.17 

.14 

Test3A 

-.06 

-.05 

-.02 

-.05 

.08 

.10 

.80 

Perf.  Hours* 

-.47 

-.38 

-.19 

-.20 

-.60 

-.31 

-.60 

-.17 

Written  Hours^ 

-.05 

-.04 

-.68 

-.43 

-.06 

-.02 

-.10 

-.05  .12 

Perf.  Tests* 

51 

.68 

.06 

.05 

,64 

.75 

.42 

.36  -.48 

-.05 

Note,  n  =  307 

®Thesc  arc  composites  based  on  communality  estimates  and  the  procedures  described  in  the  text. 
^FSG2:  Mean  of  initial  test  scores  for  Blocks  1.  2,  4.  and  6. 

^BIockSA:  Standardized  Block  5A  Performance  Test  -  standardized  Block  5A  hours 
‘*Block2:  Standardized  Block  2  Written  Test  -  standardized  Block  2  hours 
"BlockSB:  Standardized  Block  SB  Performance  Test  -  standardized  Block  SB  hours 
^BIock3A:  Standardized  Block  3A  Performance  Test  -  standardized  Block  3A  hours 
*Pcrf.  Hours:  Sum  of  hours  for  Blocks  3A,  5A,  and  SB. 

^Writ.  Hours:  Sum  of  hours  for  Blocks  1«  2,  4,  and  6 

^Perf.  Tests:  Mean  of  initial  test  scores  for  Blocks  3A,  SA,  and  SB. 


(-.99)  with  FactorZ,  followed  by  TestSA  (-.84).  Block2  correlates  highest  (.97) 
with  Factors,  followed  closely  by  Test2  (.92).  Both  BlockBB  and  TestSB  correlate 
.95  with  Factor4,  followed  by  the  mean  of  the  performance  tests  (.72).  BlockSA 
correlates  highest  (.85)  with  FactorB,  followed  by  TestSA  (.71). 
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These  results  suggest  that  FSG  and  the  above  test  scores  alone  may 
represent  the  dimensions  of  performance.  However,  since  FSG  contains  Test2  and 
FSG  ana  Test2  correlate  .62,  FSG  and  Test2  do  not  constitute  independent 
dimensions.  These  findings  support  using  FSG  as  an  indicator  of  performance  on 
knowledge  tests  but  suggest  using  separate  indicators  for  performance  on  each 
available  performance  test. 

Recommendations.  Researchers  should  treat  the  two  curricula  separately. 
In  the  former  curriculum,  in  addition  to  FSG,  use  performance  test  3B  and 
performance  test  5B.  In  the  current  curriculum,  in  addition  to  FSG,  use 
performance  test  3A,  performance  test  5A,  and  performance  test  5B. 
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Apprentice  Personnel  Specialist  (73230) 

Description  of  Variables.  The  Air  Force  Apprentice  Personnel  Specialist 
(APS)  (73230)  school  consists  of  7  blocks  of  instruction.  The  7  blocks  are: 
Block  I  -  Orientation,  Block  II  -  Introduction  to  Personnel  and  General 
Administrative  Procedures,  Block  III  -  Unit  Orderly  Room  and  Customer  Assistance 
Section,  Block  IV  -  Quality  Force  Section,  Block  V  -  Career  Progression  Section, 
Block  VI  -  Personnel  Utilization  Section,  and  Block  VII  -  Personnel  Data  System. 
Student  training  hours  represent  the  number  of  hours  a  student  takes  to  complete 
a  block.  Students  have  training  hours  recorded  for  each  of  the  7  blocks. 
Students  who  disenroll  from  class  receive  a  status  variable  which  describes  the 
category  of  disenrollment  (see  Appendix  B). 

Typing  Tests:  Students  must  type  a  minimum  of  15  words  per  minute  (WPM) 
with  no  more  than  three  errors  to  pass  the  Block  I  By-Pass  test  and  to  by-pass 
the  6  additional  typing  tests  in  Blocks  I-VI.  Those  trainees  who  fail  to  type 
15  WPM  with  three  or  fewer  errors  take  typing  tests  throughout  the  course  until 
they  meet  those  standards.  (Since  few  people  had  to  take  extra  typing  tests,  the 
researchers  did  not  include  these  later  scores  in  the  analyses.)  Students  must 
attain  or  retain  a  minimum  typing  speed  of  15  WPM  with  no  more  than  five  errors 
to  pass  the  final  typing  test  and  the  Air  Force  Performance  Test  (AFPT)  70. 

Written  Tests:  Test  grades  represent  the  grades  attained  on  the  final 
exams  for  Blocks  I I-VI I.  The  Block  V  exam  has  25  items;  all  the  other  exams  have 
40  items.  If  students  fail  the  final  exam,  the  school  retests  them.  The 
students  need  to  attain  a  minimum  grade  of  70%  to  pass,  except  on  the  Block  V 
exam  where  they  need  a  72%.  Some  students  receive  an  initial  test  grade  lower 
than  the  minimum  passing  grade.  These  students  must  retake  the  block  test.  All 
students  get  a  passing  score  on  the  first  retake;  however,  the  block  grade  for 
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these  students  always  equals  the  minimum  passing  score.  The  school  computes  the 
Final  School  Grade  (FSG)  as  the  arithmetic  average  (mean)  of  the  block  grades 
(Blocks  II  through  VII),  while  we  computed  FSG  using  the  initial  test  scores 
(FSG2).  The  database  included  test  grades  for  Blocks  II  through  VII.  The  school 
measures  all  test  and  block  grades  and  FSG  on  a  scale  of  0-100. 

Quizzes:  Students  must  take  from  7  to  18  quizzes  before  taking  the  final 
exam  for  a  block.  The  school  scores  the  quizzes  on  a  Satisfactory 
(S)/Unsatisfactory  (U)  basis.  If  students  fail  the  first  block  exam,  they  repeat 
the  quizzes  and  take  a  block  retest.  Numbers  for  quiz  data  represent  total 
number  of  "U"s  on  the  first  attempt.  The  quiz  variables  consist  of  the  initial 
and  retest  quizzes  shown  in  Table  130.  Since  the  researchers  entered  the  number 
of  "U"s  into  the  database,  for  the  analyses  we  recoded  these  data  to  reflect  the 
number  of  ''S"s  using  the  formula:  number  of  "S"s  *  number  of  quizzes  -  number, 
of  "U"s.  Thus,  for  example,  in  Block  II:  number  of  "S"s  -  17  -  number  of  "U"s, 


Table  130 

Apprentice  Personnel  Specialist  Initial  and  Retest  Quiz  Variables 


Block 

Quiz 

No.  of 
Quizes 

Block 

Quiz 

No.  of 
Quizzes 

II 

Initial 

17 

V 

Initial 

7 

11 

Retest 

17 

V 

Retest 

7 

m 

Initial 

18 

VI 

Initial 

16 

m 

Retest 

18 

VI 

Retest 

16 

IV 

Initial 

12 

VII 

Initial 

13 

rv 

Retest 

12 

VII 

Retest 

13 
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Sample.  We  analyzed  22  variables  after  eliminating  all  the  retest  quiz 
variables,  since  they  had  little  or  no  variability  and  values  only  for  a  very 
small  number  of  cases  (between  6  and  68  cases).  We  did  not  factor  analyze  Final 
School  Grade  since  it  consists  of  the  weighted  test  scores.  From  the  initial 
sample  of  385,  the  final  sample  analyzed  consisted  of  297  cases  with  no  missing 
data  on  any  of  the  22  variables  listed  in  Table  131. 

Results  and  Discussion.  Factor  analysis  indicates  three  factors.  Table 
132  shows  the  commonalities,  rotated  factor  loadings,  and  factor  score 
coefficients.  The  primary  factor  (Factor  1)  contrasts  training  hours  with  test 
scores  [(hours  for  Blocks  II  through  VII)  -  (test  scores  for  Blocks  II,  III,  IV, 


Table  131 

DcschDtive  Statistics  for  73230  Variables 


Variable 

Mean 

Sid  Dev 

Skew 

Kurtosis 

Minimum 

Maximum 

Block  1  By-Pass  Test  (WPM) 

28.960 

9.615 

1.158 

1.719 

15.000 

68.000 

Final  Tvping  Test  (WTM) 

32.380 

8.555 

1.593 

4.071 

20.000 

72.000 

AFPT76(WPM) 

31.428 

8.103 

1308 

3342 

20.000 

72.000 

Block  I  Hours 

5.951 

.149 

•2.725 

5.461 

5300 

6.000 

Block  II  Hours 

40.638 

10.485 

1.197 

4.069 

18.000 

92.000 

Block  III  Hours 

32.047 

13.852 

1.378 

1.852 

12300 

90300 

Block  IV  Hours 

24.634 

10.746 

1.800 

4.467 

7.300 

77.000 

Block  V  Hours 

20^97 

8.863 

1.733 

5.370 

7.300 

71.800 

Block  VI  Hours 

31^18 

11.933 

1.470 

3.013 

12300 

84.000 

Block  VII  Hours 

23.455 

6.949 

1.088 

1.973 

10.000 

54.000 

Block  II  Test 

84.613 

8.274 

-.208 

-.404 

60.000 

100.000 

Block  III  Test 

76.747 

10.445 

-.160 

-.231 

45.000 

100.000 

Block  IV  Test 

80516 

10.635 

-390 

.231 

45.000 

100.000 

Block  V  Test 

81.650 

9.711 

-.448 

-.104 

48.000 

100.000 

Block  VI  Test 

80.455 

9.179 

-.239 

-.208 

50.000 

100.000 

Block  VII  Test 

85.721 

7.726 

-.440 

-.250 

63.000 

100.000 

Block  II  Initial  Quiz 

15.485 

1.445 

-.896 

343 

10.000 

17.000 

Block  III  Initial  Quiz 

17.081 

.990 

-1.110 

1.317 

13.000 

18.000 

Block  IV  Initial  Quiz 

11.401 

.796 

-1.373 

1.727 

8.000 

12.000 

Block  V  Initial  Quiz 

6.620 

.647 

-1.695 

2.459 

4.000 

7.000 

Block  VI  Initial  Quiz 

15.027 

1.023 

-.969 

337 

11.000 

16.000 

Block  VII  Initial  Quiz 

12.273 

.868 

-1.340 

2.396 

8.000 

13.000 

Note:  N  =  297 
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Tabic  1?: 

CommunaiiMc?;.  f-acior  Loadings,  and  Factor  Score  Coefficients  for  *3230  Variables 


Vanablc 

Communal  ity 

Factor  Loadings 

Factor  Score  Coefficients 

Fac  1: 
Hours 

Fac  2: 
FSG 

Fac  3: 

WTM 

Fac  1: 
Hours 

Fac  2: 
FSG 

Fac  3: 
WPM 

Block  1  By-Pass  Test 

.793 

-.099 

.109 

.878 

-.013 

-.005 

.137 

Final  Tvping  Test 

.946 

-.062 

.114 

.964 

.073 

.061 

.600 

AFPT7b 

.888 

-.109 

.082 

.933 

-.061 

-.092 

.289 

Block  I  Hours 

.007 

.021 

.002 

.080 

-.001 

-.002 

.000 

Block  II  Hours 

.459 

.650 

-.155 

-.111 

.179 

.066 

.013 

Block  III  Hours 

^37 

.725 

-.105 

.031 

.276 

.127 

.001 

Block  IV  Hours 

SS6 

.729 

-.155 

-.020 

.283 

.120 

.021 

Block  V  Hours 

.378 

.603 

-.120 

-.010 

.090 

.006 

-.019 

Block  VI  Hours 

.496 

.697 

-.097 

.040 

.240 

.088 

.032 

Block  VII  Hours 

.387 

.571 

-.246 

.015 

.100 

-.026 

.017 

Block  II  Test 

.453 

-.472 

.479 

.037 

-.055 

.132 

.000 

Block  HI  Test 

.438 

-.478 

.457 

.014 

-.031 

.142 

-.029 

Block  IV  Test 

.439 

-.409 

.513 

.095 

-.014 

.170 

.102 

Block  V  Test 

.233 

-.161 

.444 

.097 

-.015 

.098 

-.026 

Block  VI  Test 

.273 

-.360 

.378 

-.002 

-.010 

.081 

-.008 

Block  VII  Test 

.330 

-.171 

.549 

-.001 

.020 

.165 

-.022 

Block  II  Initial  Quiz 

.394 

-.255 

.573 

.010 

.035 

.208 

-.020 

Block  III  Initial  Quiz 

.236 

-.161 

.457 

-.039 

.021 

.118 

-.018 

Block  IV  Initial  Quiz 

.240 

-.103 

.473 

.077 

.038 

.144 

-.009 

Block  V  Initial  Quiz 

.117 

.016 

.328 

.096 

.028 

.082 

-.009 

Block  VI  Initial  Quiz 

.274 

-.118 

.509 

.026 

.042 

.164 

-.017 

Block  VII  Initial  Quiz 

.204 

-.012 

.450 

.022 

.060 

.145 

-.003 

Note.  N  =  297 


plus  VI)].  The  second  factor  (Factor  2)  defines  a  "test  plus  quiz"  factor  (the 
sum  of  all  test  scores  plus  all  quiz  scores),  while  the  third  (Factor  3)  defines 
a  "typing"  or  WPM  factor  (the  sum  of  Block  I  By-Pass  Test,  the  Final  Typing  Test, 
and  AFPT70). 

Table  133  displays  descriptive  statistics  on  composite  variables  derived 
from  the  factors.  The  sum  of  training  hours  and  total  WPM  each  had  relatively 
large  standard  deviations,  indicating  large  individual  differences  among  the 
students.  The  distributions  of  the  criteria  show  only  a  slight  positive  skew. 

Table  134  shows  the  correlations  among  the  factors  and  composites.  Since 
FSG  and  FSG2  correlate  .98,  this  offers  no  support  for  using  initial  test  scores 
to  compute  FSG. 
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Tabic  133 

Descnnive  Statistics  for  73330  Composite  Variables 


Compos  lie 

Reliability^ 

.Mean 

Sid  Dev 

Skew 

Kunosis 

Minimum  Maximum 

Hours* 

.84 

172.889 

47.121 

.644 

.460 

71.100 

349.000 

FSG 

S/A 

82^12 

5.672 

.558 

-.098 

70.000 

100.000 

FSG2' 

.77 

81.684 

6.391 

.281 

-.243 

66.333 

99.667 

WTM‘* 

.95 

30.923 

8.375 

1.445 

3.237 

18333 

70,333 

*These  arc  composites  based  on  commonality  estimates  and  the  procedures  described  in  the  text. 

^Hours:  Sum  of  Hours  for  Blocks  II  through  VII 

^FSG2:  (Sum  of  Initial  Test  Scores  for  Blocks  II  through  VlI)/6 

'*WPM:  (By-Pass  +  Final  +  AFPT70)/3 


Table  134 

Correlations  Among  73230  Composites  and  Factors 


(Composite 

Factor  1 

Factor  2 

Factor  3 

Hours 

FSG 

FSG2 

Factor  2 

-.13 

Factor  3 

-.00 

.03 

Hours® 

.98 

-.21 

-.01 

FS 

-.50 

.SO 

.08 

-.52 

FSG2° 

-36 

.79 

.06 

-37 

.98 

WPM' 

-.10 

.12 

.98 

-.11 

.20 

00 

*Hours:  Sum  of  Hours  for  Blocks  II  through  VII 

^FSG2:  (Sum  of  Initial  Test  Scores  for  Blocks  II  through  VlI)/6 

^WPM:  (By-Pass  +  Final  +  AFPT70)/3 


Hours  (the  unit-weighted  sum  of  student  training  hours  for  Blocks  II 
through  VII)  correlates  highest  (.98)  with  Factor  1.  Such  a  high  correlation 
between  the  unit-weighted  sum  of  training  hours  with  the  empirically  defined 
Factor  1  supports  use  of  the  simpler  unit-weighted  sum  of  training  hours.  The 
sum  of  standardized  training  hours  for  Blocks  II  through  VII  and  empirical  Factor 

I  correlate  .97  (not  shown  in  Table  134).  These  two  correlations  actually  exceed 
those  for  the  unit-weighted  hours-test  score  contrast  with  the  empirical  Factor 
1.  Using  raw  scores,  the  unit-weighted  hours-test  score  contrast  and  Factor  1 
correlate  .94,  and  using  standard  scores,  they  correlate  .91  (not  shown  in  Table 
134).  All  these  results  support  using  just  the  sum  of  training  hours  for  Blocks 

II  through  VII  as  a  meaningful  criterion  dimension. 
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FSG  shows  an  inverse  relationship  both  with  Factor  1  (r=-.50)  and  with  the 
sum  of  hours  for  Blocks  II  through  VII  (r=-.52).  This  measure  of  training  hours 
indicates  that  students  who  spent  more  time  in  training  tended  to  have  lower 
FSGs;  weaker  students  needed  to  practice  more  to  get  through  the  APS  school 
program,  while  students  with  higher  FSGs  spent  less  time  in  training. 

Factor  2  consists  of  the  sum  of  the  test  grades  plus  the  sum  of  the  initial 
quiz  grades.  Due  to  the  similarity  of  this  factor  to  FSG  (the  average  of  the 
test  grades  for  Blocks  II  through  VII),  the  researchers  examined  whether  FSG 
could  sufficiently  represent  this  dimension.  FSG  correlates  highest  (.80)  with 
Factor  2,  followed  closely  by  FSG2  (.79).  The  unit-weighted  test-plus-quiz  sum 
and  Factor  2  correlate  .82  (not  shown  in  Table  134).  While  the  correlation 
between  the  unit-weighted  test-plus-quiz  sum  and  Factor  2  slightly  exceeds  that 
for  FSG  and  Factor  2,  the  school  already  uses  FSG  and  FSG  represents  a  useful 
summary  measure.  The  sum  of  the  quizzes  correlates  .89  with  Factor  2  and  .59 
with  FSG  (not  shown  in  Table  134).  Thus,  the  quizzes  compose  some  aspect  of 
Factor  2,  independent  of  the  test  grades,  but  their  scale  of  measurement  differs 
from  that  of  the  test  (and  block)  grades. 

WPM  (the  average  of  Block  I  By-Pass  Typing  Test,  Final  Typing  Test,  and 
AFPT70  Typing  Test)  correlates  .98  with  Factor  3  using  either  raw  or  standard 
scores.  Therefore,  the  average  of  the  typing  test  scores  represents  a  meaningful 
criterion  dimension. 

These  composites  have  the  advantage  of  not  overlapping  with  any  other 
composite;  variables  on  any  one  composite  do  not  appear  on  any  other  composite. 
This  approach  creates  conceptually  independent  dimensions. 

Recommendations.  In  addition  to  FSG,  use  the  sum  of  student  training  hours 
for  Blocks  II  through  VII  and  the  mean  of  typing  tests  (WPM). 
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SUMMARY  AND  DISCUSSION 


Table  135  summarizes  the  recommended  criterion  measures  in  addition  to  FSG 
for  the  27  curricula  investigated.  For  8,  we  recommend  only  a  single  overall 
criterion  (i.e.,  either  FSG  or  some  final  average).  For  9,  we  recommend  2 
criteria.  For  9  schools,  we  recommend  3  criteria;  and  for  1  school,  we  recommend 
4  criteria. 

Notice  that  the  training  for  seven  different  specialties  had  more  than  one 
curriculum  ongoing  during  data  collection  for  this  project.  For  Navy  ET,  these 
represent  two  separate  courses:  (a)  Electronics  Technician,  Phase  I,  ET(I),  and 
(b)  Electronics  Technician.  Phase  II,  ET(II).  For  Army  IIH  (Heavy  Antiarmor 
Weapons  Crewman),  the  two  curricula  represent  ongoing  parallel  curricula  with  a 
common  initial  component.  For  Army  19K  (Tank  Crewman),  the  four  curricula 
basically  represent  instructor  freedom  to  reorder  training  content  and  tests. 
In  addition  to  the  changes  in  the  19K  school,  four  other  schools  (i.e.,  a  total 
of  5)  implemented  changes  to  the  curricula  during  data  collection. 
Investigations  of  Curriculum  Changes 

During  the  course  of  this  study,  5  of  the  18  schools  (or  27.8%)  did  change 
their  curricula.  For  19K,  differences  amounted  to  instructor  freedom  to  order 
curricula  and  tests.  For  AV,  the  school  kept  the  same  content  areas  but  changed 
its  instruction  in  1  the  of  6  areas.  The  AV  school  also  changed  lab  scoring  to 
pass/fail.  For  MM,  the  school  integrated  the  curriculum  from  the  feeder  school 
(PE)  with  their  curriculum  and  began  using  their  own  military  instructors  instead 
of  civilians.  For  OS,  the  school  reordered  parts  of  the  curriculum  and  ceased 
its  practice  of  including  quiz  scores  in  the  FSG.  For  27230,  the  school  moved 
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STUDENT  ACTION  CODES 


B-0 


student  Action  Codes 


Appendix  B  lists  and  defines  student  action  codes  for  the  three  services 
in  the  study.  Student  action  codes  indicate  the  reasons  a  school  dropped  a 
student  from  training. 

Table  B-1 

student  Action  Codes  (Navy) 


CODE 

DEFINITION 

ACTION* 

A 

ACADEMIC  SETBACK  PREFIX 

ATU 

Possible  typographical  error  (AMS  school) 

N 

F 

ACADEMIC  ATTRITION  PREFIX 

FAC 

Lack  of  reading  skills 

R 

FAF 

Lack  of  math  skills 

R 

FAG 

Lack  of  comprehension/retention  of  material 

R/T 

FAJ 

Lack  of  comprehension/retention  of  material 

R 

FAM 

Lack  of  language  proficiency 

R 

FCD 

Lack  of  manual  ski 11  s/dexterity  in  use  of  tools 

R/T 

FCG 

Lack  of  knowledge  application 

R/T 

FCJ 

Lack  of  knowledge  application 

R 

G_ 

NON-ACADEMIC  ATTRITION  PREFIX 

GA 

MOTIVATIONAL  ATTRITION 

GAA 

Disinterest/Negative  military  attitude 

R/T 

GAD 

Disinterest/Dropped  at  student’s  request 

R/T 

GAG 

Lack  of  performance/Not  school  of  choice 

R/T 

GAJ 

Lack  of  performance/Not  school  of  choice 

R 

GAM 

Lack  of  performance/Not  what  expected 

R 

GAN 

Lack  of  performance/Negative  training  attitude 

R/T 

GAQ 

Lack  of  performance/Negative  training  attitude 

R 

GB 

ADMINISTRATIVE  ATTRITION 

GBA 

Alcohol  rehabilitation 

R/T  ■ 

GBB 

Alcohol  rehabilitation 

D/S 

GBG 

Hardship 

R/T 

GBH 

Hardship 

D/S 

GBJ 

Hardship 

R 

"Note:  Action  indicates  the  student’s  future  status  after  attrition. 

R/T  Reassigned/Transferred 

D/S  Discharged/Separated 

R  Reclassified 

N  Not  Defined 
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Table  B-1  (cont’d) 


CODE  DEFINITION 


ACTION* 


I 

I 


I 


GC_  THRU  MEDICAL  ATTRITION 
GG_ 

GCB  Pregnancy 

GCE  Orthopedic/Service  connected 

GCG  Orthopedic/Pre-Service 

GCH  Orthopedic/Pre-Service 

GCL  Podiatry/Service  connected 

CDS  Neurology/Service  connected 

GDY  Dermatology/Pre-Service 

GEB  Internal  Medicine/Service  connected 

GEC  Internal  Medicine/Service  connected 

GEE  Internal  Medicine/Pre-Service 

GEL  Ear,  Nose,  Throat/Pre-Service 

GEM  Ear,  Nose,  Throat/Pre-Service 

GEV  Psychiatric/Service  connected 

GEY  Psychiatric/Pre-Service 

GFB  Psychiatric  (Suicidal )/Service  connected 

GFE  Psychiatric  (Suicidal )/Pre-Service 

GFH  Psychological  (Personal ity  Disorders) 

GFL  Psychological  (Enuresis) 

GFP  Psychological  (Sleepwalking) 

GFQ  Psychological  (Sleepwalking) 

GFR  Psychological  (Situation  reaction) 

GGA  Other  medical/Service  connected 

GGB  Other  medical/Service  connected 

GGC  Other  medical/Service  connected 

GGD  Other  medical/Pre-Service 

GGE  Other  medical/Pre-Service 


OTHER  NON  ACADEMIC  ATTRITION 
GHA  Legal  (Arrest  by  civil  authorities) 

GHE  Legal  (Civil  conviction) 

GHN  Legal  (Misconduct) 

GHP  Legal  (Misconduct) 

GHR  Legal  (Substance  abuse) 

GHS  Legal  (Substance  abuse) 

GHV  Homosexual i ty 


D/S 

D/S 

R/T 

D/S 

D/S 

D/S 

D/S 

D/S 

R 

D/S 

D/S 

R 

D/S 

D/S 

D/S 

D/S 

D/S 

D/S 

D/S 

R 

R/T 

R/T 

D/S 

R 

R/T 

D/S 


R/T 

D/S 

R/T 

D/S 

R/T 

D/S 

D/S 


“Note:  Action  indicates  the  student’s  future  status  after  attrition. 

R/T  Reassigned/Transferred 

D/S  Discharged/Separated 

R  Reclassified 

N  Not  defined 


] 
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Table  B-1  (cont’d) 


CODE  DEFINITION  ACTION® 


GOB  Death/Non-Training  related  D/S 

GKL  Obesity  D/S 

GLH  Fraudulent  Enlistment  (Drug  subsequence)  D/S 

GLS  Fraudulent  Enlistment  (Arrest  pre-service)  D/S 

GMB  Erroneous  Enlistment  D/S 

GNE  Other  Non-Academic  D/S 

GUR  Legal  (Declared  deserter)  D/S 

H_  ADMINISTRATIVE  DISENROLLMENT  PREFIX 

HAC  Cancellation  of  class/course  R 

HBC  Rating  or  program  conversion  R 

HCA  Change  in  student’s  orders  R/T 

HCB  Change  in  student’s  orders  D/S 

HCC  Change  in  student’s  orders  R 

HDA  Failure  to  meet  prerequisites  R/T 

H“C  Failure  to  meet  prerequisites  R 

HDD  Possible  typographical  error  (EN  and  ET  schools) 

HDJ  Possible  typographical  error  (ET  school) 

HEA  Failure  to  meet  security  requirements  R/T 

HEB  Failure  to  meet  security  requirements  D/S 

HEC  Failure  to  meet  security  requirements  R 


ZXX  Administrative  drop/Restart  at  a  later  date 

ZRD  Completed  training 


“Note:  Action  indicates  the  student’s  future  status  after  attrition. 

R/T  Reassigned/Transferred 

D/S  Discharged/Separated 

R  Reclassified 

N  Not  Defined 
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Table  B-2 

Student  Action  Codes  (Air  Forcp) 


Air 

Traffic  Controller  (272301 

Personnel  Soecialist  (73230) 

CODE^  DEFINITION 

1 

CODE*  DEFINITION 

LA 

Academic  Deficiency 

I  Incomplete 

LG 

Separated  Service 

LI 

Misconduct 

U  Unsuccessful 

LJ 

Entry  into  the  Military  Academy 

LL 

Death  -  Training  Related 

LM 

Medical 

LP 

Prerequisite  Deficiency 

LQ 

Death  -  Other 

LR 

Performance  Deficiency 

LS 

Security 

LT 

Administrative  Reasons 

LU 

Unsuitability 

LV 

Compassionate 

LW 

Excessive  Absence  (including 
AWOL) 

LX 

Other 

“Air 

Force  codes  result  in  the  student’s 

>  immediate  elimination  from  training. 

Table  B-3 

Student  Action  Codes  fArmvl 


CODE* 

DEFINITION 

1 

Medical  Reasons 

2 

Failed  to  Qualify 

3 

Discharged 

4 

Unknown 

“Army  codes  result  in  the  student’s  immediate  elimination  from  training. 
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APPENDIX  C 

AVIATION  ELECTRICIAN’S  MATE  (AE)  TEST  WEIGHTS 
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Aviation  Electrician’s  Mate  (AE)  Test  Weights 


Table  C  -  1 

Aviation  Electrician’s  Mate  (AE)  Test  Weights 


Test 


Weight 


Knowledge  Progress  Test  (module  #301)  100% 

=  Unit  0  Grade 

Knowledge  Progress  Test  (module  #311)  100% 

=  Unit  1  Grade 

Performance  Progress  Test  (module  #322)  30% 

Knowledge  Progress  Test  (module  #321)  70% 

=  Unit  2  Grade 

Performance  Progress  Test  (module  #332)  30% 

Knowledge  Progress  Test  (module  #331)  70% 

=  Unit  3  Grade 

Performance  Progress  Test  (module  #342)  30% 

Within-course  Comprehensive  Test  I  (module  #341)  70% 

»  Unit  4  Grade 

Performance  Progress  Test  (module  #352)  30% 

Knowledge  Progress  Test  (module  #351)  70% 

*  Unit  5  Grade 

Knowledge  Progress  Test  (module  #361)  100% 

>  Unit  6  Grade 

Performance  Progress  Test  (module  #372)  40% 

Knowledge  Progress  Test  (module  #371)  60% 

«  Unit  7  Grade 

Performance  Progress  Test  (module  #382)  40% 

Knowledge  Progress  Test  (module  #381)  60% 

■  Unit  8  Grade 
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Table  C  -  1  (cont’d) 


l 


[ 

I 


1 


Test  Weight 


Performance  Progress  Test  (module  #392)  10% 

Performance  Progress  Test  (module  #393)  10% 

Performance  Progress  Test  (module  #394)  10% 

Performance  Progress  Test  (module  #395)  10% 

Performance  Progress  Test  (module  #396)  10% 

Within-course  Comprehensive  Test  II  (module  #391)  50% 

=  Unit  9  Grade 

Performance  Progress  Test  (module  #402)  10% 

Performance  Progress  Test  (module  #403)  10% 

Performance  Progress  Test  (module  #404)  5% 

Knowledge  Progress  Test  (module  #401)  75% 

=  Unit  10  Grade 

Performance  Progress  Test  (module  #412)  40% 

Knowledge  Progress  Test  (module  #411)  60% 

*  Unit  11  Grade 

Performance  Progress  Test  (module  #422)  30% 

Knowledge  Progress  Test  (module  #421)  70% 

«  Unit  12  Grade 

Performance  Progress  Test  (module  #432)  5% 

Performance  Progress  Test  (module  #433)  10% 

Performance  Progress  Test  (module  #434)  10% 

Performance  Progress  Test  (module  #435)  5% 

Performance  Progress  Test  (module  #436)  10% 

Knowledge  Progress  Test  (module  #431)  60% 

=  Unit  13  Grade 

Performance  Progress  Test  (module  #442)  50% 

Within-course  Comprehensive  Test  III  (module  #441)  50% 

■  Unit  14  Grade 

Knowledge  Progress  Test  (module  #451)  100% 


=  Unit  15  Grade 


t 


C-2 


1 


Table  C  -  2 


Aviation  Electrician’s  Mate  (AE)  Unit  Weights 


Unit  Weight 


Unit  0  Grade  3% 

Unit  1  Grade  5% 

Unit  2  Grade  6% 

Unit  3  Grade  6% 

Unit  4  Grade  8% 

Unit  5  Grade  7% 

Unit  6  Grade  5% 

Unit  7  Grade  5% 

Unit  8  Grade  7% 

Unit  9  Grade  10% 

Unit  10  Grade  6% 

Unit  11  Grade  8% 

Unit  12  Grade  7% 

Unit  13  Grade  5% 

Unit  14  Grade  10% 

Unit  15  Grade  2% 

*  Final  School  Grade 
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APPENDIX  D 


AVIATION  ORDNANCEMAN  (AO)  TEST  WEIGHTS 
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Aviation  Ordnanceman  (AO)  Test  Weights 


Table  0  -  1 


Test  Weight 


Curriculum  A: 


Mean  of  all  Practical  Works  20% 

Mean  of  all  Knowledge  Progress  Tests  80% 

*  N 

N  90% 

Final  Comprehensive  Knowledge  Test  10% 

=  Final  School  Grade 


Curriculum  B: 


Mean  of  all  Practical  Works  10% 

Mean  of  all  Knowledge  Progress  Tests  90% 

=  N 

N  70% 

Within-course  Comprehensive  Knowledge  Test  10% 

Within-course  Comprehensive  Performance  Test  10% 

Final  Comprehensive  Knowledge  Test  10% 

=  Final  School  Grade 
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APPENDIX  E 


AVIATION  TECHNICIAN  (AV)  CHANGES  AND  TEST  WEIGHTS 


E-0 


Aviation  Technician  (AV)  Changes  and  Test  Weights 


Before  June  28,  1991  (Julian:  91178),  the  school  weighted  the  course  lab 
scores  and  included  these  scores  in  the  FSG.  On  this  date,  the  school  changed 
its  scoring  method  for  labs  to  satisfactory/unsatisfactory  and  no  longer  included 
lab  scores  in  the  FSG.  Because  of  these  changes,  the  school  revised  its  test 
weighting  scheme.  Below,  we  provide  both  weighting  systems  (before  and  after  the 
changes  of  June  28,  1991).  In  both  tables,  the  school  provided  RGI  with  the 
school  assigned  weights  and  we  calculated  the  contribution  of  each  test  to  the 

f'SG.  , 

Table  E-1  applies  to  students  enrolled  before  Julian  date  91178.  The  AV 
school  altered  its  curriculum  three  times  while  using  this  weighting  system.  The 
school  added  Test  700  to  Part  6,  then  added  Test  445  to  Part  4,  then  removed  Test 
500  from  Part  5.  The  present  AV  instructors  did  not  work  at  the  AV  school  at  the 
time  of  these  changes.  Consequently,  the  school  did  not  know  how  the  previous 
instructors  redistributed  the  test  weights  to  adjust  for  the  changes. 
Furthermore,  the  AV  school  records  do  not  indicate  the  effect  of  these  changes 
upon  the  test  weights.  Without  this  information,  we  developed  assumptions  for 
t^ese  weights,  based  upon  earlier  and  later  trends  in  the  school’s  methods  of 
weight  distribution.  Then,  we  randomly  selected  several  ISS  cases  and  used  a 
hand  calculator  to  confirm  the  assumed  weights.  In  Table  E-1,  we  marked  each 
assumed  weight  with  an  asterisk. 

Table  E-2  provides  the  revised  weighting  system  the  AV  school  used  for  all- 
students  enrolled  as  of  Julian  data  91178.  The  table  includes  course  #60  test 
module  numbers  in  parentheses  because  the  school  continued  to  use  the  revised 
weighting  scheme  after  changing  the  course  number  to  60  in  December,  1991. 
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Table  E  -  1 


Aviation  Technician  (AV)  Test  Weights  (For  studends  enrolled  before  June  28 
1991 


Test  or  Lab 
Ncnber 


School  Assignee 
Weight 


Contribution 
to  FSG 


PART  1 : 


Knowledge  Test  111 


100X 

=  Section  1  Grade 


2.0280X 


Knowledge  Tests  121  &  122 

Mean  Score  of  labs  109,  119,  129,  139 

Performance  Test  124 


44.4X  (22.2X  each) 
11. IX 
44. 5X 

s  Section  2  Grade 


1.9971X  each 
.9986X 
4.0032X 


Knowledge  Tests  131  &  132 
Mean  Score  of  Labs  149,  159,  169,  179 
(If  registered  after  Julian  date 
91157,  this  average  includes  Lab  189) 
Performance  Test  133 


Section  1  Grade 
Section  2  &  3  Grades 
Comprehensive  Tests  123  I  100 


44. 4X  (22. 2X  each) 
11. IX 


44. 5X 

s  Section  3  Grade 
07.8X 

69. 2X  (34. 6X  each) 
23.0X  (11. 5X  each) 
-  PART  1  GRADE 


1.9971X  each 
.9986X 


4.0030X 


2.9900X  each 


PART  2: 


Knowledge  Tests  211,  221,  241 

Mean  Score  of  Labs  219,  229,  239,  249 


Knowledge  Tests  251,  252,  &  254 
Mean  Score  of  Labs  259,  269,  279,  289, 
299,  209 

Performance  Tests  253  &  255 


Section  1-4  Grade 
Section  5  Grade 
Comprehensive  Test  200 


85.8X  (28.6X  each) 
14. 2X 

=  Section  1-4  Grade 
39.9X  (13.3X  ea.) 
06. 7X 

53.4X  (26.7X  ea.) 

=  Section  5  Grade 

28.0X 
60. OX 
12.0X 

»  PART  2  GRADE 


2.0002X  each 
.9940X 


1.9950X  each 

1.0005X 
4.0050X  each 


3.0000X 


PART  3! 


Knowledge  Tests  321,  331,  I  341 
Mean  Score  of  Labs  309,  319,  329, 
339,  349,  369 


85.8X  (28.6X  ea.) 
14.2X 

«  PART  3  GRADE 


2.0020X  each 
•9940X 


Table  E  -  1  (cont’d) 


Test  or  Lab  School  Assigned  Contribution 

Nutter  Weight  to  FSG 


PART  4; 


If  enrolled  before  91105  and  not  in 
class  number  90820,  90821,  91010, 

91011,  91140,  91141,  91280,  or  91281; 
Knowledge  Tests  431,  441,  442,  &  443 
OR 

If  enrolled  after  91104  or  in  class 
nuTter  90820,  90821,  91010,  91011, 

91140,  91141,  91280,  or  91281: 

Knowledge  Tests  431,  441,  442,  443,  &  445 
Mean  Score  of  Labs  409,  419,  429, 

439,  449,  459 
Performance  Test  444 


Sec  ion  1-4  Grade 
Comprehensive  Test  400 

PART  5: 

If  enrolled  before  90183: 

Knowledge  Tests  511  &  521 

Mean  Score  of  Labs  509,  519,  529,  539 

Performance  Tests  512  &  522 


Sections  1-3  Grade 
Comprehensive  Test  500 

OR 

If  enrolled  after  90182; 

Knowledge  Tests  511  &  521 

Mean  Score  of  Labs  509,  519,  529,  539 

Performance  Tests  512  I  522 

PART  6; 

Knowledge  Test  601 
Knowledge  Tests  600  I  700 
Mean  Score  of  labs  609  I  619 
Performance  Test  611 


61.6%  (15.4%  each)  2.0008%  each 


61.6%*  (12.32*  each)  1.6006%  each 


07.7%  1.0004% 
30.7%  3.9885% 
s  Section  1-4  Grade 


81,2% 

18.8%  3,0008% 

=  PART  4  GRADE 


30.8%  (15.4%  each)  2.0008%  each 

07.8%  1 .0134% 

61.4%  (30.7%  each)  3.9885%  each 

«  Section  1-3  Grade 


81.2% 

18.8%*  3.0080% 

B  PART  5  GRADE 


43.8%*  (21.9%*  each)  3,5040%  each 

06.2%  .9920% 

50.0%  (25.0%  each)  4.0000%  each 

-  PART  5  GRADE 


20.0% 

30.0%*  (15.0%*  each) 

10.0% 

40»0% 

>  PART  6  GRADE 


2.0000% 
3.0000%  each 
1.0000% 
4.0000% 


For  the  weighting  scheme  above,  the  contribution  of  parts  1-6  to  the  FSG  is  as  follows. 


PART  1  26% 

PART  2  25% 

PART  3  07% 

PART  4  16% 

PART  5  16% 

PART  6  10X 
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Table  E  -  2 


Aviation  Technician  (AVI  Test  Weights  (For  students  enrolled  since  June  28.  19911 


Test 

Ntnber 

School  Assigned 
Weight 

Contribution 
to  FSG 

PART  1 : 

Knowledge  Tests  111  (100),  121  (106), 

122  (115),  131  (139),  132  (K8) 

Performance  Tests  124  (127)  &  133  (160) 
Comprehensive  Tests  123  (130)  &  100  (163) 

11.10  (2.22 
08.46  (4.23 
06.44  (3.22 
=  26.0 

each) 

each) 

each) 

.5772% 

1.0998% 

.8372% 

each 

each 

each 

PART  2: 

Knowledge  Tests  211  (206).  221  (212).  241 
(221).  251  (227).  252  (252).  254  (269) 
Performance  Tests  253  (251)  &  255  (275) 
Comprehensive  Test  200  (281) 

13.32  (2.22 
08.46  (  4.23 
03.22 
=  25.00 

each) 

each) 

.5550% 

1.0575% 

.8050% 

each 

each 

PART  3: 

Knowledge  Test  321  (306) 

Knowledge  Test  331  (315) 

Knowledge  Test  341  (324) 

02.22 

02.33 

02.44 
»  06.99 

.1551% 

•1629% 

.1706% 

PART  4: 

Knowledge  Tests  431  (412).  441  (424), 

442  (433),  443  (439),  445  (448) 

Performance  Test  444  (451) 

Comprehensive  Test  400  (454) 

08.65  (1.73 
04.18 

03.18 
»  16.01 

each) 

.2770% 

.6692% 

.5091% 

each 

PART  5: 

Knowledge  Tests  511  (521)  &  521  (545) 
Performance  Tests  512  (518)  &  522  (539) 

07.50  (3.75 
08.50  (4.25 
«  16.00 

each) 

each) 

.6000% 

.6800% 

each 

each 

PART  6: 

• 

Knowledge  Test  601  (600) 

Knowledge  Tests  600  (60S)  A  700  (606) 
Performance  Test  611  (604) 

02.25 

03.50  (1.75 
04.25 
*  10.00 

each) 

.2250% 

.1750% 

.4250% 

each 

For  this  weighting  schtme, 

PART  1 
PART  2 
PART  3 
PART  4 
PART  5 
PART  6 


the  contribution  of  parts 

26.00X 

25.00% 

06.99% 

16.01% 

16.00% 

10.00% 
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1-6  to  the  FSG  is  as  follows. 


APPENDIX  F 


ELECTRONICS  TECHNICIAN  (ET)  PHASE  1  TEST  WEIGHTS 


Electronics  Technician  (ET)  Phase  1  Test  Weights 


Table  F  -  1 


Electronics  Technician  (ETl  Phase  1  Test  Weights 


Test 

Weight 

Knowledge  Test  (Area  100) 

5% 

Performance  Test  (Area  200) 

1% 

Knowledge  Test  (Area  200) 

9% 

Performance  Test  (Area  300) 

2% 

Knowledge  Test  (Area  300) 

8% 

Performance  Test  (Area  400) 

2% 

Knowledge  Test  (Area  400) 

8% 

Performance  Test  (Area  500) 

2% 

Knowledge  Test  (Area  500) 

8% 

Performance  Test  (Area  600) 

2% 

Knowledge  Test  (Area  600) 

8% 

Performance  Test  (Area  700) 

3% 

Knowledge  Test  (Area  700) 

12% 

Performance  Test  (Area  800) 

3% 

Knowledge  Test  (Area  800) 

12% 

Performance  Test  #1  (Area  900) 

2.25% 

Performance  Test  #2  (Area  900) 

2.25% 

Knowledge  Exam  (Area  900) 

10.5% 

=  Final  School  Grade 
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APPENDIX  G 

MACHINIST’S  MATE  (MM2)  CHANGES  AND  TEST  WEIGHTS 
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Table  G  -  2 


Machinist’s  Mate  (HH21  Test  Weights 


Test  or  Quiz 

Weight 

Knowledge  Test  1  (module  #305) 

90% 

Quiz  1  (module  #204) 

10% 

-  Unit  1  Grade 

Knowledge  Test  2  (module  #309) 

90% 

Quiz  2  (module  #206) 

05% 

Quiz  3  (module  #208) 

05% 

=  Unit  2  Grade 

Knowledge  Test  3  (module  #313) 

90% 

Quiz  4  (module  #212) 

10% 

-  Unit  3  Grade 

Knowledge  Test  4  (module  #317) 

90% 

Quiz  5  (module  #214) 

10% 

■  Unit  4  Grade 

Average  of  Unit  Grades  1-4 

80% 

Comprehensive  Test  1  (module  #319) 

20% 

»  Phase  1  Grade 

Knowledge  Test  5  (module  #325) 

90% 

Quiz  6  (module  #223) 

05% 

Quiz  7  (module  #224) 

05% 

-  Unit  5  Grade 

Knowledge  Test  6  (module  #328) 

90% 

Quiz  8  (module  #227) 

10% 

«  Unit  6  Grade 

Knowledge  Test  7  (module  #331) 

90% 

Quiz  9  (module  #230) 

10% 

»  Unit  7  Grade 
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Table  G  -  2  (Cont’d) 


Test  or  Quiz 

Weight 

Knowledge  Test  8  (module  #334) 

100% 

»  Unit 

8  Grade 

Knowledge  Test  9  (module  #338) 

90% 

Quiz  10  (module  #235) 

05% 

Quiz  11  (module  #237) 

05% 

=  Unit 

9  Grade 

Knowledge  Test  10  (module  #341) 

90% 

Quiz  12  (module  #239) 

10% 

■ 

=  Unit 

10  Grade 

Knowledge  Test  11  (module  #344) 

90% 

Quiz  13  (module  #242) 

10% 

-  Unit 

11  Grade 

Average  of  Unit  Grades  5-11 

75% 

Comprehensive  Test  2  (module  #349) 

20% 

Performance  Test  (module  #144) 

05% 

-  Phase 

1  2  Grade 

Phase  1  Grade 

50% 

Phase  2  Grade 

50% 

-  Final 

School  Grade  (FSG) 
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